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Some Thoughts on Direction 


For the past four years, Wally Worth has guided the fortunes of AJER. His 
legacy is a high quality, well-managed journal that serves the educational 
research community in Canada and abroad. As the new editor, I intend to 
continue in the Worth tradition of being a “hands-on” editor. This means 
that readers will see perspectives from national experts on important re- 
search issues, critical reviews of research programs, short notes on promis- 
ing research directions, and even the occasional long article on a topic which 
seems fundamental to the development of our field. It also means that 
writers may experience some frustration as I “encourage” them to make 
their work more accessible to the wider community of educational research- 
ers. 

The main goal of AJER is communication. Reports of educational re- 
search need not be soporific. Our journal (ours because it belongs to the re- 
search community) must stimulate curiosity, excitement, debate, and in- 
sight. To further this goal, I am repeating some advice offered by Bill 
Bruneau, editor of the Canadian Journal of Education, who suggested to 
his contributors that research articles should be clear, should be set in con- 
text, should be persuasive, and most importantly, should advance our un- 
derstanding of education. 

Seen from the inside, education is a broad field. Researchers are aware 
that in a country like ours the number of people who work in a particular 
area is very small. Few readers of an article will share the specific expertise 
of the writer. It is important, therefore, to place reports in context: Where 
does this research fit? Why is it important? Arguments must be carefully 
crafted and the evidence displayed for the convenience of the reader. The 
whole point of the report is to provide a coherent, structured portrayal so 
that readers can assess its significance and decide what relevance it has for 
their own work. 

If we succeed in making our work more readable, and thus more avail- 
able to scholars working in other areas of education, we will provide a stron- 
ger base for the evolution of our field. 


AOE 
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J. DAVID MacKINNON 


Acadia University 


Living with Conformity 
in Student Teaching 


Despite the criticisms which have been leveled at teacher education programs, the field ex- 
perience component of preservice preparation continues to enjoy the support of virtually all 
stakeholder groups. Indeed, it is most commonly identified as an indispensable element of 
professional preparation. Although graduates are virtually unanimous in their praise of 
student teaching, questions have been raised about the practicum experience, especially 
about the tendency of many student teachers to passively conform to the practices of their 
cooperating teachers. This paper explores the experiences of four prospective teachers of 
young children during an eight-week practicum, their reasons for conforming, and the im- 
plications their stories have for preservice teacher preparation. For these students, con- 
formity was not a matter of passive acceptance of the status quo but was rather a response 
to their interpretations of the constraints of being a student teacher. 


The importance attached to field experiences in the preservice preparation 
of classroom teachers by all stakeholder groups is a clear testament to the 
apprenticeship nature of teacher education. Cortis (1985, p. 14) has likened 
teacher education to medieval craft guilds, and the analogy, at least in its 
dominant orientation, is not far-fetched. The practicum, specifically, is con- 
sidered to be the sine qua non of teacher preparation. Lanier and Little 
(1986, p. 550) point out that while the university-based study of pedagogy is 
frequently assigned limited value, especially by practitioners, practice teach- 
ing is considered to be essential. Its importance is implicitly acknowledged 
by the sheer quantity of research which has focused on student teaching, 
while other aspects of professional preparation have received much less at- 
tention (Hersom, 1984, p. 255; Nemser, 1983, pp. 155-156). However, the 
dynamic nature of the practicum experience has received scant attention 
(Adler, 1984, p. 15). As Ryan (1980) has pointed out, “we know very little 
about the relationship of our training and what is really going on in the 
minds and emotions of our students” (p. 116). 

The purpose of this paper is to explore briefly one aspect of the prac- 
ticum experience: the oft reported tendency of student teachers to conform 
to the practices and expectations of their cooperating teachers (Beyer, 1984; 
Clifton, 1979; Dickie, 1983; Goodman, 1985; Seperson & Joyce, 1973; Tar- 
dif, 1984; Zimpher, deVoss, & Nott, 1980). In essence, it is an attempt to 
come to some understanding of the dynamics of student teaching. Why is 
this important? One could cite the relatively dismal knowledge base in 
teacher education (Ralston, 1980; Sears, 1984), the questionable impact of 
professional preparation on teaching practice (Arnstine, 1979; Lortie, 
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1975), the suggestion that the practicum is primarily a socialization (some 
have suggested an uncritical one) into the world of schools (Iannaccone, 
1963; Smith, 1980), or the alleged theory-practice dichotomy which finds 
frequent expression in the teacher education literature (Miklos & Greene, 
1987; Miller & Taylor, 1984). All of these are supportable reasons for inves- 
tigating just what is going on when education students venture into real 
classrooms with real children. 

But there is another reason as well. Tabachnick, Popkewitz, and Zeich- 
ner (1979-80) maintain that “the characteristics of the student teaching ex- 
periences are not to be found in public statements of intention but through 
an examination of the experience itself” (p. 28). It is the individual’s inter- 
pretation of his or her experience that is so important in moving toward an 
understanding of the world through another’s eyes, in determining how 
someone has understood what they have experienced. Teacher education 
programs, at the very least, are intended to have a beneficial and trans- 
forming influence on prospective teachers. Given the stated importance of 
the practicum and its potential for facilitating (or thwarting) the transfer of 
theory into practice, teacher educators should never assume that field ex- 
periences are in and of themselves worthwhile and educative. Nor should 
they assume that what is intended to happen during the practicum in fact 
happens. 

Changing orientations to educational research are now opening the 
doors to a number of studies which have as their fundamental intent an un- 
derstanding of the dynamics of student teaching (Britzman, 1986; Dickie, 
1983; Feiman-Nemser & Buchmann, 1987; Lapin, 1984; Ralston, 1980; 
Tabachnick et al., 1979-80; Tardif, 1984; Zeichner & Liston, 1987). Some of 
these studies raise serious questions about the value of student teaching as 
it is presently structured in many universities. The data presented in this 
paper are intended to contribute to our understanding of preservice field ex- 
periences and their value in preparing prospective teachers. 


The Nature of the Study 
Introduction 


The data on the practicum presented in this article derive from a larger 
study of the teacher education experience (MacKinnon, 1987). The purpose 
of this larger study was to contextualize and interpret the meaning which 
selected teacher candidates gave to their university classroom and prac- 
ticum experiences during one academic term. In order to do this I spent one 
academic term with a class of 27 third-year education students specializing 
in early childhood at a Canadian university. 

By the time the practicum began, I had already spent a number of weeks 
with the class as a participant observer. I had attended all their classes, 
taken part in all their activities (except compulsory assignments and tests), 
and spent innumerable hours with a reduced number of them exploring 
what course work in teacher education was like. (They were enrolled in two 
courses: one on program environments in early childhood education, and 
one in educational administration.) Who I was and what I was doing was 


J.D. MacKinnon 


well known to all of them; they had welcomed me into the class. Being with 
them so much of the time allowed me to be present as experience unfolded 
and they began to talk about, and react to, their day-to-day activities. It also 
placed me in a unique position to develop the solid field relationships so cru- 
cial to the success of a study of this type (Whyte, 1979, p. 59). Consequently, 
as they prepared to begin student teaching, I had already come to know 
most of the students reasonably well and had developed rapport with 
several of them. This helped me greatly when it came time to select in- 
dividuals who would become key informants during the practicum. 


A Conceptual Orientation 


As meaning was the key concept in the study, I drew on the basic tenets of 
symbolic interactionism, a major social theory which addresses the inter- 
pretation of meaning. Interactionism takes as its central focus the relation- 
ship between the individual and the larger social setting. This perspective 
essentially rests on three underlying assumptions about the individual in 
society: (a) that people act toward things based on the meaning these things 
have for them; (b) that the meaning of such things comes from the social in- 
teraction one has with others; and (c) that these meanings are handled and 
modified through an interpretive process (Blumer, 1969, p. 2). This inter- 
pretive process, conceptualized as intermediary between stimulus and 
response, is critical in symbolic communication, for the same symbol can 
have different meanings in different contexts (Ritzer, 1983, p. 163). Thus 
response is conceptualized as a constructed act based on the meaning which 
the individual has assigned through an interpretation of significant sym- 
bols. 

Among the many concepts native to symbolic interactionism, one that 
emerged as particularly relevant to the practicum experience, and is thus of 
central importance here, is “definition of the situation.” Definition of the 
situation refers to the way individuals take into account the various ele- 
ments in their immediate situations in order to decide on appropriate be- 
havior. As Lauer and Handel (1983) note, this involves representing the 
situation “to the self symbolically so that a response can be made” (p. 127). 
The significance of how individuals define situations is underscored in the 
classic assertion by Thomas and Thomas (1928) that “if men define situa- 
tions as real, they are real in their consequences” (p. 572). How situations 
are defined is influenced by a host of factors including the goals one has in a 
particular situation, the application of perspectives developed from previous 
interaction, and the view one has of oneself in that setting (Charon, 1979, 
pp. 134-138). 

Situations may also be defined intentionally. That is, individuals pursu- 
ing particular goals in given situations will attempt to influence the defini- 
tions of others. This may be achieved by what one chooses to say, by how it 
is said, by certain physical gestures, and even by choice of clothing. This is 
especially prevalent in Goffman’s (1959) dramaturgical approach to inter- 
actionism. The drama metaphor is founded on the concept of impression 
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management, that individuals attempt to guide and influence the impres- 
sions others form of them. 

For the participants in this study, the ways in which they defined the 
practicum had a significant bearing on their decisions regarding what they 
could and could not do as student teachers. 


The Fieldwork 


This paper focuses exclusively on the practicum. Student teaching occupied 
the last eight weeks of the academic term, and this particular practicum 
was especially important in the minds of all the students because they had 
been placed at their preferred level, that is, K-3. The majority, however, 
found themselves in kindergarten classrooms, with only a few placed at any 
of the other three grade levels. 

In order to make the study manageable, I focused exclusively on four 
student teachers. Three of them were in a kindergarten setting, and one in 
grade 1. Two factors influenced the selection of these students. The first in- 
volved the nature of our relationship. I selected individuals with whom I re- 
lated easily and who indicated that they would be willing to accommodate 
regular classroom visits from me. The second pertained to the nature of 
their placement. In early childhood classes at the university, a great deal of 
emphasis had been placed on the structuring of appropriate learning en- 
vironments for young children. (“Structure” was an overriding theme 
throughout the term, and it referred to the way a learner’s time is engaged 
while he or she is in the classroom or learning environment. A highly struc- 
tured classroom is one in which a student exercises little self-direction and 
most of the activities are controlled by the teacher.) The principle thrust of 
these classes had been that relatively unstructured, child-directed activities 
are more suitable than structured, teacher-directed ones. The four student 
teachers I chose to visit had been placed in classrooms in which teaching ar- 
rangements varied from relatively unstructured to tightly structured (or 
traditional) techniques. Most of these set-ups would not have been condoned 
by their early childhood instructors and I wanted to see how the students 
dealt with their situations in light of this, or whether it had an influence at 
all. 

I visited each student teacher at least once each week for a number of 
hours, and all conversations were recorded (with permission) on a pocket- 
size cassette tape recorder. The tape recorder also served other purposes. It 
was a substitute for my field book (which I had used extensively during 
university classes), thus allowing me to chronicle events or make personal 
comments about things that happened. In addition, I would talk with each 
informant during the evening after an observation in order to explore the 
day’s events more fully. These telephone calls were all recorded. 

In order to supplement the data I gathered during my visits and conver- 
sations with these four student teachers, I selected eight students to serve 
as secondary informants. I interviewed each of these student teachers once 
a week over the telephone and recorded the conversations. (In the case of 
two of these secondary informants, we met at a prearranged location every 
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week or more for two to three hours of informal discussion on their student 
teaching experience.) In two instances I made an on-site visit. The data 
gathered from these conversations were used to supplement and enrich that 
collected from the four key informants. 

In order to ensure that what I was seeing and understanding of the prac- 
ticum experience was in fact what the student teachers were experiencing, 
each was asked to critically review all quotations attributed to them and the 
context in which they were being employed. The credibility of data of this 
nature lies in determining whether categories created in data analysis are 
shared in meaning between the researcher and the participants (LeCompte 
& Goetz, 1982, p. 44). 


Data Analysis 


The general orientation which I adopted toward data analysis followed from 
a conceptualization of research as more a cognitive undertaking and less a 
validation process. Bailyn (1977, p. 97) points out that this implies data 
analysis as an ongoing, continuous activity involving a constant interplay 
between concepts and data in which the conceptual understanding of the re- 
searcher is developmental rather than assumed beforehand. Such an orien- 
tation allows continuous data analysis to guide the inquiry by moving from 
broad descriptive observations to more focused and selective ones (Spradley, 
1980). For each student teacher, I maintained a chronological record of the 
events which took place during the eight-week practicum. This record was 
composed of the numerous tapes made during school visits and evening 
telephone conversations. Fach tape was dated and marked with the name of 
the student teacher recorded thereon. In addition, each student teacher 
maintained a log of her activities during the practicum, and I was permitted 
access to these. 

The actual analysis of these data was a task far greater in magnitude 
than I had ever imagined. Although I had read numerous accounts by other 
researchers who had undertaken similar projects, their descriptions of the 
difficulties encountered held little meaning for me until I saw the amount of 
data that was accumulating. 

I attacked this task by first reviewing the suggestions made by Bogdan 
and Biklen (1982) and Turner (1981). Both offer useful advice for reducing 
large quantities of written material to a more manageable form. Turner 
(1981) sets out a nine-stage prescription for the manipulation of raw data 
into refined analysis which, although not followed precisely, was useful to 
me in the early coding and categorization of my data. I shied away, how- 
ever, from rigidly following any prescriptive account of “qualitative” data 
analysis, for this seemed too much like prestructuring what should not be 
prestructured. 

Initially I planned to have the tapes transcribed. Then I was going to 
code and analyze the transcripts in a manner similar to the way I had dealt 
with my field notes during the first half of the term. But transcribing 
proved to be both expensive and time-consuming. The time element was im- 
portant, for I needed to listen to all the tapes from one week before I visited 
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the student teachers again the next week. After a period of trial and error, I 
settled on the following system: After every visit and follow-up telephone 
conversation, | listened to the tapes and made notes on the issues and con- 
cerns identified by each individual. For each note I made, I recorded its tape 
footage so that I could easily retrieve the reference. When I finished this I 
gave each issue or concern which I had noted a descriptive label or category 
and entered each of these on its own file card. In the end, I defined over 30 
categories for each student teacher—each of which represented an issue 
that she had identified during the practice teaching experience. The whole 
process was begun anew for each individual in order to avoid the temptation 
to make the categories developed for one student teacher fit another. 

Once this was done I looked for similarities between categories for each 
individual and, as a result of apparent interrelationships, reduced the 
numerous categories into a smaller number of themes. One of these themes 
forms the basis of later discussion on the student teaching experience. 

Before presenting the data on the student teaching experience, I wish to 
provide a brief picture of the four principal participants as they began the 
practicum. Although the names have been changed, these descriptions are 
nonetheless intended to give body and context to what would otherwise be 
nothing more than a collection of accounts from faceless, nameless stu- 
dents. 


The Key Informants 
Becky 


Becky was slightly older than most of her classmates. She had been married 
and had worked at other jobs before making the decision to become a 
teacher. Nevertheless, she claimed she had always wanted to be a teacher 
and fondly recalled playing school with her younger brother when she was a 
child. 

She was placed in a kindergarten classroom in one of the area’s newer 
schools. There were two kindergarten classes each day, one in the morning 
and one in the afternoon, and each lasted for two and a half hours. Each 
class was structured in a similar manner. The children spent the first 30 
minutes in the gymnasium where they either played on their own or 
engaged in exercises aimed primarily at gross motor development. From 
there they went to their classroom for various opening exercises and 15 to 
30 minutes of teacher-directed time (“circle time”). Then came approxi- 
mately 30 minutes of “center time,” where they could play on their own at 
learning centers which had been set up around the room. The session 
wound down with a short recess and ended with a 30-minute block of teach- 
er-directed time. 

Becky was really excited about her placement. She had nothing but 
praise for the atmosphere in the school and for her cooperating teacher in 
particular. This is how she described it when I first visited her: 

It’s really neat. The whole atmosphere in the school ... the kids are free to be people. 


They’re not told to sit down or shut up, as you can tell by this room (referring to the 
flexible arrangement in the room). It’s the way a school should be—fun. It still has 
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rules, it still has structure, but it’s fun. It’s just neat to see. I hope that’s the way our 
trend is going. 


Beth 

Unlike most of the other student teachers, Beth claimed to have made the 
choice to become a teacher about two months before arriving at university. 
However, she recalled that as a child she always assumed the role of leader 
when she and her family visited her younger cousins. Throughout childhood 
and adolescence she received much positive reinforcement about her ability 
to relate to young children. 

Beth was placed in a kindergarten classroom in an inner-city Catholic 
elementary-junior high school. There were two kindergarten classes each 
day. The first 30 minutes of class began with teacher-directed opening exer- 
cises including attendance, prayer, a good morning (afternoon) song, calen- 
dar activity, and a check on the weather. The next 30 minutes was devoted 
to center time, when the children either visited learning centers of their 
own volition or engaged in a more structured activity such as an art project. 
This was followed by a snack and a 15-minute recess. The day ended with 
more teacher-directed activities such as music time, story time, and a good- 
bye song. 

Beth’s initial reaction to her placement stood in marked contrast to 
Becky’s. Her dismay can be directly traced to the university’s early child- 
hood classes she had attended. There the students had been consistently ex- 
posed to a child-centered philosophy of school curricula and pedagogy which 
taught that children should be permitted to actively engage in the learning 
process and that they are, if given the chance, capable of providing the self- 
direction to do so. Consequently, “appropriate” early childhood classrooms 
were ones which allowed for a lot of self-direction by the children and mini- 
mized the amount of time where the teacher directly controlled what was 
taking place. 

Beth was undoubtedly the most fervent champion of an open approach 
to teaching in the class, and she was astounded and disappointed to find 
herself in a kindergarten where most of the children’s time was teacher- 
directed. She was especially distressed to learn that her cooperating teach- 
er’s approach to language instruction centered on individual letters of the 
alphabet: letter days, cutting out letters, tracing letters, learning the sounds 
of the letters, and so on. She ardently believed that it was all meaningless. 
Because of this, the tone set early in the practicum was anything but posi- 
tive (although she did not openly betray her feelings to her cooperating 
teacher). This is how Beth put it: 


My initial reaction was that (in a slightly subdued voice) this is horrible. Because it’s 
very quiet in here. You know, the kids do not get overly loud and have fun and play. 


Margo 


Margo, like Becky, was slightly older than most of her classmates. She was 
a daycare worker before entering university. But like Beth, her decision to 
become a teacher could not be traced to a childhood desire. In fact, while 
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teaching was something she could see herself doing, she also had other in- 
terests: “If I never taught a day in my life it wouldn’t bother me.” 

She was placed in a kindergarten classroom in a small suburban elemen- 
tary school. There were, again, two classes each day, and the structuring of 
the school day was similar to Becky’s and Beth’s situations. The first 30 
minutes was for teacher-directed circle time. The routine was fairly struc- 
tured: a few moments with the calendar, then on to the weather; a review of 
hand signals (which told them to look or to listen), and a song or two. This 
was followed by 30 minutes of center time. The children could go to various 
centers, but there were always at least two which were compulsory (usually 
math and language). After a short recess the children went to the library 
where they could browse on their own or listen to a story. The day con- 
cluded with a 30-minute gym period. 

Margo’s belief in a child-centered approach to curriculum and instruc- 
tion was much like Beth’s. However, she had a happy-go-lucky side to her 
personality which seemed to prevent her from becoming as distressed as 
Beth over the negative aspects of her placement. She quickly adjusted to the 
fact that, although this was not how she would structure her own class- 
room, this was the way things were and there was little she could do about 
it. But she told me she was disappointed at first, and her feelings about her 
placement were unambiguous in the following log entry, written on the 
second day of the practicum: 


I feel that [this] classroom is geared for the teachers. There is not one speck of 
children’s work on any part of the classroom walls. I feel strongly that the classroom 
should be the children’s room, that anyone who would walk into the classroom would 
be able to get a feel for the children in the room because the walls would be full of 
their work (stories, pictures, writing, art, etc.). I find that this room has many lovely 
posters and pictures on the walls—but do the children notice them or make comments 
about them? Many of them are not even on the child’s eye level. How meaningful are 
these wall decorations? I know that this is something I would change in my own room. 


Melinda 


Melinda was one of the younger members of the class and had always 
wanted to be a teacher. She clearly remembered playing school as a 
youngster, and while she could not isolate a particular time when she made 
the decision to become a teacher, she felt she had always been headed in 
that direction. 

Unlike the other three, Melinda was placed in a grade | classroom in a 
Catholic elementary school, and it contrasted with the other placements in 
almost every way. Her day was much more traditionally arranged. Instead 
of preparing for a two and a half hour class, Melinda was faced with a 
school day which was divided into eight periods. While a specialist teacher 
taught music, and her cooperating teacher retained the daily religion class 
(Melinda was not Catholic), Melinda, once she assumed a full load, was 
responsible for language arts, math, science, social studies, health, art, and 
physical education. Most of these subjects occupied a 40-minute time slot 
each. 


J.D. MacKinnon 


Melinda was the only one who did not react noticeably to the structural 
elements of her placement. Whereas Margo and Beth were decidedly critical 
of their situations, and Becky ecstatic with hers, Melinda rarely mentioned 
the physical arrangement or time structuring of her grade 1 classroom. She 
was neither critical nor approving. Instead, she was more concerned from 
the beginning with gaining a sense of what her cooperating teacher ex- 
pected from her and how she would be evaluated. In her words: 


I was just more concerned with what I would be asked to do, [and] when. What was my 

teacher like? How would she evaluate me? How much flexibility would she allow me in 

planning my lessons and things like that? 

Student Teaching 

During the fieldwork and data analysis stages of the research, I developed 
eight themes on the practicum experience based on what I was seeing and 
hearing during the hours spent with each student teacher. Each theme rep- 
resented, by definition, a unifying idea running through a portion of the 
data. Specifically, each was something shared by all the student teachers, 
and while they did not necessarily experience it in a similar fashion, it was 
an issue or concern for all. These eight themes focused on (a) the issue of 
conformity (the focus of this paper); (b) the significance of the cooperating 
teacher to the student teaching experience; (c) the tensions created by 
university supervisors, especially where their expectations for the student 
teachers were at variance with those of the cooperating teachers; (d) the ex- 
perience of being evaluated by both cooperating teachers and university su- 
pervisors; (e) the importance to the student teachers of having effective 
classroom control; (f) attempts by some of the student teachers to subtly 
change some of the structural arrangements in their classrooms; (g) the 
tendency to view student teaching as a performance; and (h) the distinc- 
tions made by many of the student teachers between the “real” world of the 
classroom and the “ideal” world of the university. 

A dominant theme, that is, one that arose again and again in conversa- 
tion with the student teachers, was their sense of having to conform to the 
expectations (stated or perceived) of their cooperating teachers. All of them, 
some more than others, were critical of the pedagogy they saw practiced in 
their classrooms. Most would have liked to try out some of the approaches 
advocated in the early childhood classes they took at university, and they 
claimed that if they were in their own classrooms, things would be different. 
But they were not in their own classrooms, and they did not try out many 
new ideas. On the surface, it looked as if the hours spent in classes had had 
little effect. What follows is a glimpse below the surface. It is an attempt to 
understand why they conformed and how they felt about doing what they 
did. And in the telling, their stories raise questions about what purpose a 
practicum is intended to serve in the preparation of classroom teachers. 


Role Definition: A Tendency to Conform 


A question of status. Guinea pigs: That was how Margo described stu- 
dent teachers during the first few days of a practicum. She felt that because 
they were new on the job the children would try to test them, and because 
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the student teachers did not yet know the rules of the school and the class- 
room, they had to fumble along as best they could. 

From the beginning Becky, Beth, Margo, and Melinda found that their 
status as student teachers was really not a status at all. The classrooms 
they entered were not theirs. As such, they were guests, and they saw them- 
selves as being subservient to their cooperating teachers. This meant hold- 
ing in check all the impulses and beliefs which might clash with what they 
defined as the existing norms of their situations. Margo told me early in the 
practicum that she had no choice about doing certain things in her room. I 
asked why. 


(She laughs.) I just wouldn’t go and say: “I’m totally against it. I’m not going to do it.” 
You have to realize that I’m a student teacher. I’m not running the classroom. You 
don’t go into the class and start dictating what you like and what you don’t like. 
There’s a lot of things I would do differently but that doesn’t mean I can change it. 


Margo was the best at articulating what it was like to be a student 
teacher, although each of the others told the same story in different words. 
It was like being expected to be a teacher and a student at the same time: a 
teacher to the children, but a student to the cooperating teacher. But, for 
Margo at least, the student role dominated. As she said: “I think until I’m 
not a student I’ll feel like a student.” Near the end of the practicum she 
wrote in her log: “I still don’t feel 100% comfortable at all times. Perhaps 
because I know that I am playing the role of a student teacher.” And even 
when she did feel certain of herself in her opposition to some aspect of class- 
room practice, she usually chose to comply rather than make an issue of it. 
“You just kind of learn to bite your tongue or play along. It’s not worth the 
time and effort to make a big deal out of things that I don’t feel comfortable 
with.” 

She wasn’t alone in this understanding of what it was to be a student 
teacher. Melinda shared her views, and although it snowballed to the point 
of causing her a great deal of irritation, she felt powerless to do anything 
about it: 


[Sometimes] I feel frustrated and restricted and ... well, not overwhelmed ... just get- 
ting tired of people telling me what to do. Just having people above you all the time, 
you know, it’s not like it’s your own classroom. Sure I’m going to be [student] teaching 
full time, but it’s not my classroom. There’s rules that have been established in there 
that I don’t feel I have any right to change. 


Beth also felt this sense of powerlessness in the way she defined the role 
of student teachers. And, given her dedication to unstructured learning en- 
vironments, this was a bitter pill to swallow. Her classroom was, in her 
view, anything but unstructured. Nevertheless, she felt she had no option: 

You’ve got to do what the teacher says. You’re stuck. You can’t be your own teacher; 

you’ve got to do what the teacher says so that you’re not looking like you’re coming in 

here and saying: “Well, your program sucks. I’m doing it my way.” 

So compliance was, in the eyes of these student teachers, a taken-for- 
granted part of being an outsider in someone else’s classroom. 

Many of the secondary informants concurred. One told me how frus- 
trated she was at having to conduct lessons which focused on individual let- 
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ters of the alphabet (a no-no according to their early childhood instructors) 
and frequently referred to them as “stupid.” But she didn’t voice her com- 
plaints to her cooperating teacher: “I just do what she tells me to do and 
everything’s OK.” And when I would question this approach, she would tell 
me that that was what her teacher had planned for the year, and that’s the 
way it was. It was as if there were no other possible answer, as if my ques- 
tion were silly. Another said of her situation, referring to her cooperating 
teacher, “It is her class.” And yet another: “It’s her classroom. I’m just the 
student teacher. I’m there to learn from her.” Everyone said the same 
thing. 

The impact of change. But there was more to conformity for student 
teachers than simply defining their status as being subservient to their 
cooperating teachers. The impact of any potential change on the children 
was also on the minds of many of them. I first became aware of it in conver- 
sation with Becky: 

I like to pay attention to how she [the cooperating teacher] deals with the kids. After 

you choose the teacher, she’s the boss. The way she does it is the right way as far as 

these little kids are concerned. It’s the only way they’ve ever seen. So you basically 
have to model your teaching in accordance with this. If it’s your classroom and you set 

up the routine, the rules and the way you teach with the kids, they’re used to you. So 

you have to respect that and you can’t go doing things differently. So I like to watch 

and listen to the way she talks with the kids and pick up on how she deals with them 


because they are comfortable with that, and I don’t want to make them feel uncom- 
fortable. 


At first I wondered whether this was little more than a way of rationaliz- 
ing compliance with established procedure. But all of them told me of in- 
stances where children were quick to speak out when their student teacher 
inadvertently strayed from the standard routine. On some occasions I ob- 
served this myself. The children in Melinda’s grade 1 classroom told her 
how she was supposed to mark their assigned work, and Becky’s kindergar- 
ten children informed her how “show and tell” was to be conducted. Beth 
told of mistakenly changing the pattern of morning exercises one day by 
starting the morning song before saying the prayer. The children inter- 
rupted her to say that things were not done that way. She tried to save face 
by asking if it was all right to change the pattern for just this one day. Most 
went along, but even then some adamantly began reciting the prayer during 
the song. She did not deviate again. 

In addition, Beth was concerned that change would lessen the children’s 
self-esteem. She told of doing a phonic routine with the class where she 
showed the letter of the day to the children and they made the sound. It 
concerned her that they frequently made the sound before the letter was 
visible. She thought of changing the order but was fearful that the children 
would think that what they thought they knew yesterday they did not know 
today. So the impact of change was a concern to her. 

Evaluation. The most frequently mentioned reason for conforming to es- 
tablished practice, however, was the fact that all of them were being 
evaluated by their cooperating teachers. Long before the practicum began, 
each of the student teachers “knew” that it would be the make or break 
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component of their four-year BEd program. This had been well ingrained 
through discussions among peers, reports from others who had gone before, 
advice from experienced teachers, and comments by their university in- 
structors. In fact, once their placements had been posted, one of their early 
childhood instructors announced in class: “Your student teaching is what 
they look for in job placements.” This was not news to anybody. 

Beth was the most ardent champion of the early childhood “philosophy.” 
There was no doubt in her mind that learning experiences had to be struc- 
tured in a meaningful way for young children. To her, this meant unstruc- 
turing them, that is, allowing the children to assume a significant amount 
of control over what they did during the school day. Knowing this, I asked 
her why she abided by the structural arrangements present in her class- 
room: 


Because they’re evaluating you. If they don’t like you they’re not going to give you a 
good evaluation no matter how good you are. That’s the sad truth but I think that is 
the truth. You talk to people who’ve gone through it before and people say: “Well, I’ve 
had a conflict of interest with my teacher.” And it shows on their final evaluation. 


Margo agreed: 


Having people observe, or knowing you’re being evaluated holds me back, I know it 
does. Because I know if I did it I would do things the way J felt right and there would 
be no one questioning me. I think that makes a difference. Until I’m not a student, I 
fee] like a student. 


In her eighth week, Margo’s frustration was plainly evident: “Let me teach 
one whole week just for myself. I don’t want anyone observing.” 

Becky’s situation was somewhat different. Her classroom came closer to 
being an unstructured setting than any of them. Furthermore, she had 
developed a warm and friendly relationship with her cooperating teacher. 
So on the rare occasions when she found herself opposed to some detail of 
pedagogy, she chose not to confront her teacher. But she made it abundant- 
ly clear that even if she was opposed to most of what her cooperating 
teacher was doing, she would play along. A good evaluation was an impor- 
tant part of the student teaching experience. 

Evaluation was mentioned frequently by many of the secondary inform- 
ants. I met with two of these student teachers one weekend for a three-hour 
conversation about their experiences. At one point during our discussion, 
one of them referred sarcastically to a lesson she had conducted on the let- 
ter N: “I did it exactly as she [her cooperating teacher] did it.” Her friend, 
who was seated across the table from her, made a symbolic gesture which, 
she explained, meant “brown-noser.” The other responded: “Listen, I want 
to get a good mark. I don’t have any choice.” 


The Effect of Conforming 

Conformity did not mean an abandonment of their own beliefs about ap- 
propriate teaching styles. Beth referred to student teaching as a “bending to 
mold yourself into the mold that the teacher has.” But it had an impact. As 
the practicum wore on, Margo found herself becoming more and more 
resigned to doing just as her cooperating teacher asked without even so 
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much as a murmur of discontent. During my last visit in her eighth week 
she told me of the details of a particular instructional unit she was doing. 
She said that her cooperating teacher had suggested a number of ideas that 
she might incorporate into the unit. Margo commented, almost smugly, 
“Well, you know what I’m going to do? Exactly that! Because I know that’s 
what she expects.” 

Not surprisingly, Beth had the greatest feelings of constraint regarding 
her situation. She was decidedly unhappy with what she saw happening to 
herself as the weeks passed. During the fifth week I asked her if she had ex- 
perienced any conflict situations since my last visit. She replied: 


The usual. Doing things I don’t like doing, like doing the letter R. I was thinking this 
morning that I came in just like a robot and changed the letter of the week, and got 
out the stuff and thought about how I was going to do the letter R. And it didn’t even 
strike me as: “God, this is horrible,” until after a while I thought: “What am I doing?” 
I’m not even thinking about it anymore. I’m into this rut where I’m doing it, and it 
scares me. 


Conformity was, simply put, a fact of life for the student teachers 
throughout the eight-week practicum. Whether for reasons of status, or out 
of concern for the children, or as a result of a pragmatic self-interest in a 
good evaluation, all of the student teachers defined the practicum as a 
situation where significant change was not an advisable course of action. 
This meant following the established schedule, maintaining the existing 
structure, and even, in some instances, adopting certain mannerisms of 
their cooperating teachers. Put another way, student teaching simply did 
not provide the opportunity for these prospective teachers to try out many 
of the ideas and skills they had learned at university. None of them would 
argue that the experience was a waste of time (although Beth came close), 
but all of them viewed the practicum as an artificial teaching experience in 
some ways. With the possible exception of Becky (who had an exemplary 
relationship with her cooperating teacher), none felt that she could truly be 
herself as long as she was constantly being scrutinized and asked to perform 
in a setting that was not hers. There was some evidence of overt contradic- 
tion, usually slight structural alterations by the student teachers in favor of 
a more child-centered approach to learning and teaching. But for the most 
part, these changes took the form of seemingly innocuous deletions or inser- 
tions in an existing routine. Perhaps Margo was not being too harsh when 
she described student teachers as guinea pigs. 


Discussion 

As I mentioned at the beginning of this paper, conformity is not an un- 
known aspect of student teaching. In a study of 10 student teachers at a 
large American university, Goodman (1985) found that the majority of 
them “passively” accepted the traditional curriculum and standard forms of 
instruction. He does refer to one particular individual, whom he describes as 
“intellectually alive and socially aware,” who sought out a cooperating 
teacher who would let her experiment during practice teaching (p. 45). But 
even here the student felt compelled to adhere to certain aspects of the 
prescribed curriculum. 
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Beyer (1984) also comments on the tendency of student teachers to abide 
by existing instructional arrangements, linking it to a desire on behalf of 
the students to succeed as professionals. But he worries that this desire, 
coupled with a cooperating teacher who reinforces compliance with the 
status quo, may lead to “uncritical uniformity” (pp. 36-37). 

And in a study of four second-language student teachers, Tardif (1984) 
noted their tendency to conform to standard practice in their classrooms by 
referring to it as “taking the path of least resistance” (p. 152). It is clear, 
however, that they disapproved of some of what they did and did it only be- 
cause to do otherwise might jeopardize success during the practicum. She 
quotes one of her informants: 


All the cooperating teachers I had were very different and I think I did well with all of 
them. The reason I did well was because I made a point to adapt myself to them and 
their situation. I think you have to do that as a student teacher. If you’re not prepared 
to do that then be prepared to have a tough time. (p. 152) 


This was essentially what Beth was saying when she spoke of “molding 
yourself” to the cooperating teacher’s ways of doing things. But the stu- 
dents in my study did not take the “path of least resistance” simply because 
it was the easiest way to go. Nor did they “passively” accept the traditional 
curriculum and standard forms of instruction, although if one were to judge 
solely on the basis of overt behavior, it would appear that way. And there 
was no “uncritical uniformity,” although Becky’s situation approached it, 
but I think this was so because her classroom was less structured than the 
others and thus more in line with what she considered to be an appropriate 
learning environment. On the contrary, most of these student teachers, and 
especially Beth and Margo, were highly critical of what they witnessed 
during the practicum. But the significance of the practicum to their teach- 
ing careers, at least as they defined it, all but precluded any attempts to do 
anything other than what the cooperating teacher specified. For them, the 
practicum was not an opportunity to try out techniques they had spent 
hours discussing and on which they had prepared assignments and presen- 
tations at university. It was, instead, a time for observation and replication. 


Rethinking Student Teaching 


It has been two years since I conducted the fieldwork for this study. Three 
of the four early childhood students are now practicing teachers. Becky and 
Beth are teaching kindergarten, and Melinda is teaching grade 1. Margo has 
not decided on a definite career move yet. She is an adventurer, and at the 
time of this writing is doing just that—having adventures—in a foreign 
land. But when she returns, assuming a teaching position is a real pos- 
sibility. 

I do not know what impact their student teaching experiences will have 
on them as full-time teachers. If their comments and criticisms from the 
practicum are accurate indicators of future actions, many will attempt to 
incorporate into their instructional repertoires much of the child-centered, 
child-directed orientation advocated in university classes and to minimize 
the more structured activities they noted during the practicum. Yet some 
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researchers have pointed out that the “liberalized” attitudes of education 
students during their preservice years is little more than a temporary adop- 
tion of university ideology which is quickly replaced by deeply-ingrained 
traditional perspectives once that experience ends (Lortie, 1975; Shipman, 
1967). Nonetheless, Zeichner and Tabachnick (1981, p. 10) contend that 
enough questions have been raised about this scenario to warrant some 
skepticism about the supposed lack of impact of university coursework. 

At the beginning of this paper I pointed out that field experiences occupy 
a prominent place in the preservice training of teachers. Even though 
teacher education is currently coming under regular attack, particularly in 
the United States, most critics agree that whatever else might be dispen- 
sable, practica are not (Cruickshank & Armaline, 1986, p. 35). But what is 
their purpose? If it is simply to acquaint prospective teachers with current 
practice, as Beyer and Zeichner (1982, p. 19) contend is frequently the case, 
then what is currently being practiced in many universities is probably ade- 
quate. Ratzlaff and Grimmett (1985, p. 27), however, infer that if this is the 
case, if professional preparation is to be merely an uncritical, passive so- 
cialization into the world of teaching, then teacher education might best be 
removed from academic settings and accomplished through field-based pro- 
grams at the hands of experienced teachers. 

If, on the other hand, practica are meant to reinforce the transfer of 
theory into practice, then the nature of that practice must come under care- 
ful scrutiny, and the experience must be structured to maximize the ac- 
complishment of this goal. In my particular study, the early childhood 
instructors at the university hoped that the practicum would provide an op- 
portunity for their students to try out much of what they had learned. This 
did not happen, and so regardless of whatever else may have been ac- 
complished by this process, the fundamental purpose of the field experience 
was undermined. 

There are, of course, other things that student teaching does. Cruick- 
shank and Armaline (1986, p. 36) list many of them including providing an 
opportunity for education students to receive feedback on their perfor- 
mance, adding realism and meaning to preservice preparation, offering an 
opportunity for novices to gauge whether they have selected the appropriate 
career, and affording an occasion for students to learn the “elements of a 
profession.” But all of this could be accomplished by some form of appren- 
ticeship program that has no connection whatsoever with a university. The 
richness of the university experience comes from repeated exposure to 
theoretical and conceptual issues, and the raison d’étre of an undergraduate 
school of education within a university is to apply current theoretical think- 
ing to educational concerns and to demonstrate their application to prac- 
tice. If any component of a teacher education program erodes this basic 
function, it needs to be either restructured or eliminated. 

There is a problem, however, if we end our discussion on this note. Sure- 
ly a mindless application of instructional strategies learned in university 
classes would be as counterproductive as an uncritical replication of current 
practice. Every effort should be made to place student teachers in situations 
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where they will be encouraged to experiment with new techniques and to 
explore different ways of approaching problems which arise in class. This 
means, among other things, that they must have a solid grounding in cur- 
ricular and pedagogical theory. But the task, as Craib (1984, p. 5) points out 
in a discussion of social theory, is less one of learning theories and more one 
of learning to think theoretically. He likens this to “learning a new lan- 
guage in a particularly difficult way, not by building up vocabulary and 
learning the various grammatical rules, but by listening to the language 
being spoken, in all its complexity, its slang, dialects, etc.” (p. 5). As I see it, 
this sort of systematic and rigorous approach to understanding social 
phenomena shares some similarities (although it is not the same thing) with 
what Morgan (1983, p. 374) refers to as “critical reflection,” where the in- 
dividual (in our case, the student teacher) is encouraged and assisted to ex- 
plore the theories and assumptions which underlie a particular action, the 
consequences of that action, and alternative ways of acting. This may be a 
particularly promising avenue of exploration for making the student teach- 
ing experience an educative one. For while critical reflection is unlikely to 
overcome conforming tendencies among student teachers, encouraging and 
assisting students to carefully analyze the assumptions which underlie 
classroom actions and to consider alternative instructional approaches may 
go a long way toward ensuring that conforming actions as student teachers 
do not follow them into their own classrooms. 

There is evidence that practica are being rethought and restructured 
along these lines in an attempt to make the student teaching experience an 
educative one. Zeichner and Liston (1987) provide an excellent account of 
one program in the United States that builds on the concept of “reflective 
action.” Its essential purpose is to assist prospective teachers in seeing the 
assumptions that underlie educational practice, and to provide an environ- 
ment where they are encouraged to do much more than simply replicate ex- 
isting practice. The student teachers are expected to live within the 
frameworks already in place in the classrooms where they teach, but they 
are also expected to explicate basic assumptions about teaching and learn- 
ing that are inherent in school activities, to develop plans for modifying and 
going beyond specified materials, to maintain a journal, and to meet for 
weekly seminars at the university to assess and explore what they are 
seeing and doing in the schools. The authors make it clear that there are 
many hurdles to overcome, but the program, if nothing else, is a move in the 
direction of guiding (and this means far more than just supervising) the 
student teaching experience, rather than assuming that a field experience 
will be beneficial, ipso facto. 

Beyer (1984) describes another program at an American university that 
ostensibly shares much in common with the one described by Zeichner and 
Liston (1987). The emphasis again is on critical reflection as a means of as- 
sisting students in seeing beneath the surface of what happens in schools. 
The benefit of this approach, as the authors point out, is that “the involve- 
ment in critical reflection and field experience ... goes some way toward 
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redirecting the tendencies of utilitarian teaching perspectives, excessive 
realism, and uncritical replication” (p. 40). 

The point of these two examples, and a lesson to be learned from the ex- 
periences of the participants in this study, is that the student teaching ex- 
perience should be as much a part of university-based, university-controlled 
teacher education as taking a course in early childhood or educational ad- 
ministration. The kinds of activities described by Zeichner and Liston 
(1987) are an example of this. If field experiences are not clearly defined 
this way in the minds of all concerned, if they are understood to be in some 
way divorced from university work, then the notion of transferring current 
theory into practice becomes little more than empty words, and the prac- 
ticum opens itself to being a miseducative experience at best. 
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Spelling Errors: 
Grade 1 to University— 
Analysis of Process and Product 


Research in spelling is often focused on either spelling products or spelling processes. 
Likewise, the common view of spelling is dichotomous: a word ts either spelled correctly or 
incorrectly. To assess the relationship of process and product and to identify developmen- 
tal trends in spelling, the unconventional spellings which occurred in 200 short essays 
written by students in grades 1, 4, 8, 10, and first-year university were examined and 
categorized by strategy employed (process) and error type (product). Analyses of strategy 
and error type by grade level suggested that (a) the categories employed could be used to 
reliably analyze spelling behavior; (6) there is a notable developmental trend in both 
strategy and error type; (c) certain orthographic patterns are particularly difficult for 
learners at various stages; and (d) employment of certain strategies results in specific types 
of errors. Findings indicate that there is a strong process/product relationship. 


Introduction 

Historically there have been two major approaches to the investigation of 
spelling. One approach has centered on the strategies employed by spellers; 
the other, more common, approach has focused on the error patterns evi- 
dent in written samples. Although researchers using the process or the 
product approach have generally mentioned the relationship of process and 
product, little has been done to systematically study the combination of the 
two. 

Teachers and the public have usually viewed spelling as a simple 
dichotomous process; the spelling of a word was either correct or incorrect. 
That is, only the products of attempts at spelling were considered. However, 
more recent research in spelling has focused on process, that is, the place 
that spelling holds in the child’s overall attempt to learn language. Seen 
from this perspective, spelling ability grows as the result of the application 
of certain semiotic principles, as noted below, in the child’s attempt to com- 
municate. 

The semiotic view proposes that spelling is a single case of a more 
general process—the use of a symbol system to convey meaning. There are 
essentially two parts to this view of the acquisition of early spelling com- 
petence: development of an awareness of the nature and use of symbols and 
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the subsequent use of articulatory features for selection of symbols. In the 
first phase children recognize the operation of symbol systems by noticing 
that symbols are carriers of meaning in the world around them. At this 
stage, children will use any symbol in their attempts to convey meaning 
(Harste, Burke, & Woodward, 1983). This interest in the use of symbols 
soon grows to include an awareness that the graphic symbol system—con- 
ventional orthography—is such that certain symbols are conventionally 
used to represent certain sound values. In the next phase, youngsters begin 
to attempt to combine their knowledge of the articulation of speech sounds 
and the names of letters used to represent these sounds in their writing 
(Read, 1975). 

Ehri (1980) describes the sophistication of this knowledge based on ex- 
amples reported by Read: 


The pre-school child is very sophisticated in analyzing a word’s phonetic structure and 

in inventing clever and plausible ways of representing that structure orthographically. 

In fact [he or she] appears to be better at this than adults who have not been trained 

in phonetics. The pre-school orthographer’s spellings change systematically as [he or 

she] gains in orthographic spelling. (p. 325) 

Spelling is one part of the semiotic process in which the child attempts to 
convey meaning through the use of conventional symbols. Within this 
process, spelling competence appears to be developmental; that is, there are 
distinct stages in the learner’s approximations of conventional spelling 
(Beers & Henderson, 1977; Zutell, 1980). Invented spellings appear to re- 
flect youngsters’ categorization of speech sounds based largely on artic- 
ulatory features of these speech sounds. As well, spelling entails both a pro- 
cess, the employment of strategies, and a product, the written word. 

The objective of this research was to replicate and expand Read’s (1975) 
and Bebout’s (1985) studies and to incorporate the semiotic view presented 
in the work of Harste et al. (1983). Whereas Read and Harste et al. focused 
on the spelling patterns and strategies of young children, this research was 
designed to investigate the developmental changes in these patterns from 
first grade to first-year university. In addition, the results of students’ spell- 
ing attempts were analyzed to provide data on the products of these spelling 
strategies. Process and product were then compared. 

In summary, the study addressed five basic questions: 


1. Are the findings of Read, Harste et al., and Bebout replicable? 

2. How do the spelling strategies and spelling products of students change 
over time? 

3. Do these changes provide evidence of developmental trends? 

4. Are particular types of words especially difficult for learners at various 
stages? 

5. What are the relationships between spelling processes and spelling 
products? 


Background to the Study 
A review of the related literature indicates that spelling has only recently 
become an area of major interest to researchers in the language arts. This 
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current interest is largely focused on the process of learning to spell, that is, 
on youngsters’ use of invented spellings and the strategies employed by pre- 
school and primary-grade children. 

Beers and Henderson (1977) and Gentry (1978, 1984) have identified 
and described the developmental aspects of youngsters’ growth in spelling 
competence. Gentry (1984) compared these stages of oral language acquisi- 
tion and defined the stages as precommunicative, semiphonetic, phonetic, 
transitional, and correct. 

Frith (1980) found that a majority of the errors produced by 12-year-olds 
who were poor spellers but good readers were phonetic misspellings. Frith’s 
analysis of these misspellings led to a description of three stages in the pro- 
cess of spelling: correct analysis of speech sounds; conversion of phonemes 
to graphemes; and selection from all the phonetically plausible graphemes. 
Frith’s description of these stages in the development of abilities in spelling 
supports the view of the interaction of process and product in the develop- 
ment toward the use of conventional orthography and guided the analysis of 
spelling in this study. 

Read (1975) investigated the phonetic stage of spelling ability and de- 
scribed the categorization of speech sounds by four- to seven-year-olds. He 
provided evidence for youngsters’ use of articulatory features in the cate- 
gorization and representation of speech sounds in spelling. 

A majority of the studies of process and product in children’s spelling, in- 
cluding those cited above, have been conducted in experimental settings. 
Harste et al. (1983) attempted to overcome the limitations of the experi- 
mental paradigm by studying youngsters’ spelling as it occurred in natural 
writing situations. This qualitative approach, in which the subjects served 
as informants, provided additional insight into the strategies employed in 
the spelling process. Harste et al. identified six spelling strategies: spelling 
the way it (1) sounds, (2) articulates, (3) looks, or (4) means; (5) spelling in 
the way similar problems have been solved; and (6) spelling by the child’s 
own or someone else’s rules. 

There are a number of areas in which the strategies identified by Harste 
et al. and the patterns described by Read overlap. Both identified strategies 
related to spelling sounds using similar letter names and spelling based on 
articulatory features. Read developed a variety of articulatory subcatego- 
ries, for example, tense/lax vowels and preconsonantal nasals, while Harste 
ei al. grouped ali the spelling errors based on articulation into a single 
category. 

In contrast to the work of Gentry, Beers, and Henderson, Read, and 
Harste et al., which focused on the spelling process, Bebout (1985) inves- 
tigated error types, that is, products, generated by learners who spoke 
English as their primary language and those for whom English was a 
second language. Although Bebout’s primary purpose was to compare the 
error patterns of the two groups, her findings related to the native English 
speaking group, thus her categorization schemes were useful in this study. 

The eight major categories of errors identified by Bebout are (1) con- 
sonant-doubling errors, (2) other consonant errors, (3) errors involving 
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schwa, (4) errors involving silent e, (5) other vowel errors, (6) letter misor- 
dering, (7) unclassifiable items, and (8) homophones. 

Anderson (1985) provided a detailed overview of recent research into the 
development of spelling strategies and stated that “Children constantly 
revise their rule systems.... Their strategies ... change as they gain more 
control of written language and become more aware of spelling principles 
and conventions” (p. 141). These strategies were described as changing 
from attempts to spell based on perceived graphophonic relationships to 
greater reliance on semantic or structural analogies to the use of lexical and 
visual information. 

The preponderance of evidence in studies conducted over the last decade 
supports the view of spelling as a developmental process, with similarities of 
stages, strategies, and patterns across large numbers of children in their at- 
tempts to use English orthography to convey meaning. This evidence was 
used as the basis for seeking developmental patterns in this study. 


Design 
The Research Model 


The research design for this study can be described as a one-shot case study 
(Campbell & Stanley, 1963). In this design each group is studied only once 
subsequent to some agent or treatment presumed to have brought about the 
observed behavior. In the present study the treatment was the subjects’ pre- 
vious learning history in spelling, and the observed behavior was the kind of 
spelling errors made in the course of writing compositions. 


Subjects and Procedures 


Short essays were collected from students in each of grades 1, 4, 8, 10, and 
first-year university. The university samples were acquired from University 
of Victoria students attending introductory composition classes. First-grade 
samples were chosen from a large collection of writing done by students in 
the Victoria School District. The grades 4 through 10 samples were random- 
ly selected from a large number of samples collected throughout the 
province as part of the British Columbia Writing Assessment. These essays 
were designed to assess the writing competence of students and taxed them 
to do their best work in terms of content, style, and mechanics including 
spelling. Students were encouraged to use the most effective words possible 
to express their ideas; consequently, they appeared willing to use words they 
could not spell correctly. This approach was assumed to provide the re- 
searchers with the broadest possible sample of spelling attempts. One pur- 
pose of this study was to assess spelling strategies employed in actual 
writing activities rather than in artificially constructed activities in which 
the spelling of specific words is targeted. Fifty samples of writing from each 
grade level, for a total of 200 samples, were analyzed. 

All spelling errors from each sample were listed and categorized for in- 
clusion in selected categories established by Read (1975), Harste et al. 
(1983), and Bebout (1985). Comparisons were made across spelling strat- 
egy, spelling strategy by grade level, and between spelling strategy and er- 
ror type. All analysis of spelling errors into error categories and strategies 
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was done by the research assistant, a doctoral student, and the principle in- 
vestigator. Interrater reliability was established at .91. 

Determination of the strategies used was done by examining and cat- 
egorizing the types of errors found in the misspelled words on the assump- 
tion that these categories reflect the students’ approaches to orthographic 
representation. In other words, if students appeared to make an error based 
on the sound of the word in their attempts to spell it, it was assumed that 
the use of graphophonic relationships was one of their spelling strategies. 
The strategy categories identified for examination were taken from the 
work of Harste et al. (1983), Gentry (1978, 1984), and Read (1975) and in- 
clude spelling a word by (1) how it sounds, (2) how it is articulated, (3) its 
appearance, (4) its meaning, (5) analogy with other spellings, and (6) own 
or learned rules. In addition, two error categories were included: a gram- 
matical error category and a handwriting error category. 


Results 
Spelling Strategies Analysis: The Spelling Process 


The presentation of findings is descriptive and focuses on comparisons 
within and between grades of the spelling strategies employed and between 
strategies and error types. 

An analysis of the graph (Figure 1) illustrating the relative reliance on 
the various strategies identified by Read-(1975) and Harste et al. (1983) in- 
dicates that whereas students at all levels rely on a variety of strategies, 
students in first grade and first-year university each show greater emphasis 
than other levels on the use of a particular strategy. First-grade students 
exhibited a strong reliance on the articulatory features of words in their 
spelling attempts. First-year university students, on the other hand, were 
found to rely more heavily on the use of spelling rules than did the younger 
students. 

It is instructive to note the error profiles within each grade level. Errors 
based on point of articulation account for over half (56.19%) of the nonstan- 
dard spellings used by the first-grade subjects. The next largest categories of 
nonstandard spellings for first-grade subjects are spelling based on the way 
a word looks (14.95%) and on the way the youngster has solved similar 
spelling problems, for example, the combination of smaller words to spell a 
larger word as in “sell fish” for “selfish.” Clearly, the major influence on 
these subjects’ spellings is point of articulation, which would indicate that 
use of articulatory features is a strategy which is relied on heavily by 
children in the early primary grades. 

The error profile for subjects in fourth grade is similar in overall shape to 
that of first-grade subjects, with point of articulation appearing as the most 
commonly employed strategy (28.53%) followed by strategies based on the 
way the youngster has solved similar problems (19.21%) and on the way the 
word looks (17.23%). 

The eighth grade marks a turning point in the strategies employed for 
spelling, as evidenced by a change in the error profile. Subjects at this level 
employed the strategy of spelling based on the way a word looks in 28.53% 
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Figure 1: Use of spelling strategies by grade 


of the nonstandard spellings. The increased reliance on this strategy 
(14.95% in first grade, 17.23% in fourth grade, 28.53% in eighth grade) and 
the concurrent decrease in reliance on articulatory features (56.19% in first 
grade, 28.53% in fourth grade, 19.03% in eighth grade) appear to indicate a 
developmental shift toward greater awareness and use of the visual aspects 
of the orthographic system in solving spelling problems. 

The error profile of subjects in tenth grade differs from that of eighth 
grade, with the greatest reliance still being on spelling based on the way a 
word looks (25.93%), followed by the way the word sounds (20.74%), and 
then by articulatory features (17.04%). Students at this level appear to be 
maintaining their use of both visual strategies and articulatory/phonological 
strategies. Use of rule-based strategies shows a slight but notable increase 
from just over 4% in grades 1, 4, and 8 to nearly 10% in tenth grade. 

This slight shift can be seen as the beginning of a second major change, 
as the profile for first-year university students shows greatest reliance on 
strategies based on rules (24.56%) and on the way a word looks (24.56%), 
closely followed by use of articulatory features (22.81%) and the way the 
word sounds (19.30%). 

In comparing the grade-level profiles, certain trends become apparent. 
Between first grade and first-year university, use of strategies 1 (the way it 
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sounds), 3 (the way it looks), and 6 (my rules or someone else’s) increases. 
Strategy 2 (the way it articulates) drops off sharply between first and fourth 
grades, and, to a much lesser degree, between fourth and eighth grades. 
Overall, the most interesting points of change are the shifts away from 
reliance on articulatory features by the end of the primary years, toward 
greater use of visual features around the eighth grade, and the increasing 
incorporation of rule-based strategies between tenth grade and first-year 
university. 

The lists of words which were spelled in nonstandard ways were ana- 
lyzed in an effort to identify spelling patterns which were particularly prob- 
lematic for students at various grade levels (Table 1). Computer programs 
based on Read (1975) were developed to identify and list nonstandard spell- 
ings of the following orthographic patterns: blends/clusters (...rm, ...spl...); 
sonorant consonants (...tle, ..son); diphthongs (...oy, ...oi...); vowel-control- 
led r/er (...ar, ...er); s/z sibilants (...us..., ...is...); digraphs (sh..., ch...); precon- 
sonantal nasals (...nt, ...ng). 

The most apparent information gained from this analysis is that there 
are many nonstandard spellings which are not accounted for by these error 
categories. The 10 categories contain between 21% and 30% of the nonstan- 
dard spellings for the subjects in the study. There are, however, a few inter- 
esting results based on the error patterns and totals found in each grade 
level. Diphthongs and vowel controlled r/er accounted for over half of the 
items listed in these 10 categories for first-grade students. Although these 
two types continued to provide some difficulty through eighth grade, other 
categories were more prominent after first grade. Fourth grade students ex- 
hibited greater difficulty with digraphs than with any other category. Also, 
this level had a considerably greater number of digraph errors than did any 
other level. 

Subjects in the eighth grade had a surprisingly high number of precon- 
sonantal nasal errors. Despite the overall drop in assessed reliance on ar- 
ticulatory features and Read’s description of preconsonantal nasal errors as 
evidence of such a strategy, the greatest number of eighth-grade subjects’ 
errors (as represented by these 10 orthographic categories) occurred in the 
preconsonantal nasal category. Despite the marked reliance on articulatory 
features by the first-grade subjects, preconsonantal nasal errors accounted 
for only 12% of the errors in the 10 categories, while 20% of the eighth- 
grade errors in this analysis were in the preconsonantal nasal category. 
Tenth-grade students exhibited greatest difficulty with blends, as did the 
first-year university subjects. The latter also had difficulties with the vowel 
controlled r/er. 


Error Category Analysis: The Spelling Product 


In her 1985 study of spelling errors made by learners of English as a first 
and as a second language, Bebout devised a comprehensive error categoriza- 
tion system. Her categorization scheme employs eight categories. This sys- 
tem was used in the present study because of its comprehensiveness and its 
focus on English orthography and also as a contrast to the strategies ap- 
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proach being employed. This procedure permitted the researcher to com- 
pare results from two classification schemes, that is, a process orientation 
(the Harste et al. and Read categories) and the product orientation (the 
Bebout categories). 

Table 2 presents the results of the overall categorization of errors where 
the spelling errors were placed in one or more of the general error cate- 
gories. The information on multiple errors consists of those words contain- 
ing two different categories of errors. 

Distinct and dramatic changes in the kinds of spelling errors made by 
student writers occurred over the 13-year sample. First-grade students 
seemed to find vowels the most troublesome orthographic feature (28.9%). 
To this might be added the schwa (11.8%) and the “silent e” (15.3%) which 
are separately coded vowel error categories. As well, their spelling was 
categorized as unanalyzable more often than any other group (18.9%). This 
finding can be coupled with the incidence of multiple errors (21.5%), again 
the highest for the total sample. 

Fourth-grade students had much more frequent difficulty representing 
consonant sounds than the first-grade students (34.2% and 14.9%), while 
their vowel problems were somewhat less (21.4% to 28.9%), and their un- 
analyzable spelling and multiple errors decreased (12.4% from 18.9%, and 
14.3% from 21.5%). Problems with the schwa decreased sharply (3.2% from 
11.8%). 

Eighth-grade students showed an error distribution very similar to those 
of the fourth grade with a further decrease in unanalyzable spellings (from 
12.4% to 8.8%) and in multiple errors (from 14.3% and 12.1%). 

Subjects in the tenth grade showed fewer errors in consonants than the 
eighth grade (25.3% to 32.7%) but an increase in unanalyzable spelling 
(from 8.8% to 10.6%). Otherwise their spelling products were similar to 
those of the fourth and eighth grades. 

The first-year university subjects exhibited remarkably different spelling 
products from the other subjects. They had much more trouble with con- 
sonant doubling than the next highest groups—the eighth and tenth grades 
(26.8% to 10.6%), much less trouble with other consonants than the tenth- 
grade group (17.9% to 25.3%) and more “other vowel” trouble than even the 
first-grade subjects (36.6% to 28.9%). On the other hand, their unanalyzed 
spelling was the lowest for all subjects (6.3%). 

The results of the present study contrast sharply with results obtained 
by Bebout in regard to the occurrence of consonant and vowel errors. Sub- 
jects in this study had relatively more trouble with consonants than did 
Bebout’s subjects except in the first grade and at university level. 


Comparison of Process and Product Analyses 


For the first-grade students it appeared that point of articulation of a sound 
is the most important spelling strategy employed. At the same time, an 
analysis of the orthographic features underlying their spelling errors 
showed that they have the greatest difficulty with both vowels and “vowel- 
like” representations (schwa and silent e). The orthographic representation 
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of the “open” articulation of the vowel sounds appeared to be most prob- 
lematic for beginning spellers. For English speakers the differences in artic- 
ulation between different vowels are less extreme than the differences in 
articulation between consonants. Thus the letters used to represent vowels 
stand for less distinctive differences than is the case for consonants. These 
physiological, phonetic, and orthographic factors may engender the ob- 
served spelling behaviors of primary students. 

Fourth-grade students, while sharing point of articulation as an impor- 
tant strategy with the first-grade subjects, now looked to analogy and ap- 
pearance as valuable spelling strategies. Their spelling products still showed 
difficulty with vowels and vowel-like sounds, but consonants emerged at 
this time as a major spelling problem. Presumably, analogy and appearance 
provided insufficient specific information about consonant representation in 
spelling at this level, due, perhaps, to lack of literary experience. 

Eighth-grade subjects (like the fourth-grade students) had begun to 
move strongly toward the use of visual strategies, for example, appearance 
and analogy, to support their use of articulation and sound values in solving 
spelling problems. This behavior resulted in spelling products similar to 
those of the fourth-grade subjects, with consonants emerging as a major 
spelling problem. The reasons for this outcome appeared to be similar to 
those for the younger group and showed that eighth-grade subjects had yet 
to master certain rules and principles of spelling. 

Tenth-grade students, while generally maintaining their use of sound, 
articulatory, and appearance strategies, but lessening their use of analogy, 
appeared to be finally moving toward the use of rules and principles in spell- 
ing. This move was also seen in the general increase in the use of or- 
thographic patterns such as digraphs and those representing diphthongs as 
shown in Table 2. The increased use of rules appeared to be one of the 
clearest signs of maturity in spelling behavior and was related to the spell- 
ing products of tenth-grade students in that their error pattern showed a 
wide distribution across error types. They seemed to have mastered many of 
the sound symbol relationships and the rules that govern English orthog- 
raphy; thus their errors reflected the general possibilities for error rather 
than focusing on certain types, thereby suggesting lack of knowledge of 
other possibilities. 

At the university level, subjects continued to use sound, articulatory, and 
appearance information, but they moved dramatically toward the use of 
rules and principles in spelling. This shift seemed to generate certain spell- 
ing product problems in that this group produced a greater proportion of 
consonant doubling errors (specifically doubling when no doubling is re- 
quired in nonaffix spellings) than any other group, as well as vowel 
problems other than the schwa and silent e. It appeared that these students 
were applying doubling rules too frequently, perhaps having been alerted to 
the presence of this rule. Similarly, they may have been relying on rules for 
the representation of vowels more than on articulatory information with 
the consequent possibility of error. They had, however, apparently mastered 
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the various representations of the schwa sounds and the rules for the use of 
silent e. 


Research Questions 


Answers to the five general questions that guided the design of this study 
summarize the results of this research: 


1. 


Are the findings of Read, Harste et al., and Bebout replicable? 


The clear answer is yes. Strategies identified by Read and Harste et al. 
and error categories developed by Bebout were capable of being reliably 
used to analyze spelling behavior. Results were generally comparable 
with those of the originators of the schemes with some variation, due 
presumably to the nature of the subjects and the source of spelling er- 
rors—connected discourse rather than individual words. 


. How do the spelling strategies and spelling products change over time? 


Spelling strategies of subjects changed from an emphasis on the sound 
and articulatory features of words to a broad-based approach using a 
variety of strategies but with special growth in the use of spelling rules. 
Products changed from greater difficulty with vowels to difficulties with 
consonants to a general range of errors sampling each of the possible 
error types. Subjects seemed to master first one then another aspect of 
English orthography as they developed. 


. Do these changes provide evidence of developmental trends? 


The clear answer here again is yes. Subjects appeared to move toward a 
broad based approach to the use of spelling strategies and errors general- 
ly distributed over the whole range of possibilities reflecting the statisti- 
cal possibility of such errors. 


. Are particular types of words especially difficult for learners at various 


stages? 

Specific difficulties include the problems of first-grade children with 
silent e, diphthongs, and vowel controlled r; consonant doubling 
problems of older subjects; and the reemergence of problems with vowel 
controlled r by university level students. 


. What are the relationships between spelling processes and spelling 


products? 


An emphasis on the use of certain strategies such as sound and articula- 
tion appears to lead to spelling errors wherein those strategies are 
manifested. The broader the range of strategies employed, the more 
general the distribution of spelling errors. 


Conclusions 


The results of this study confirm that students proceed systematically to 
develop systems or strategies for representing the sounds of words. This 
systematic process is seen as a semiotic one where language users learn to 
employ symbols (in this case orthographic symbols) in a conventional man- 
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ner. The learner establishes the essential semiotic relationship early and 
subsequently refines it. This semiotic relationship is the connection between 
the arbitrary signs of the English alphabetic system and the sound values 
associated with spoken words. It is evident from the study that only the 
simplest sort of relationships are developed in the early stages of spelling 
acquisition and that these relationships are later broadened to include a 
range of connections between symbol and sound. These varied and complex 
connections result in a number of strategies used to correctly spell a word. 
Students thus learn to replace a sound in language with a particular symbol 
or combination of symbols. The development of this ability appears to 
proceed much like the development of any other language ability in that the 
learner identifies basic principles and subsequently elaborates them. 

Spelling strategies appear to grow from an emphasis on the sound values 
and the appearance of a word to the use of this information in conjunction 
with rules, principles, and generalizations that govern English orthography. 
The nature of individuals’ spelling products (their error patterns) reflects 
this change. Students, therefore, appear to need exposure to language in its 
oral and written form, and to the rules that describe standard orthography. 
In view of the considerable emphasis on the latter in the elementary school 
and the lack of use of this strategy until the tenth grade and the university 
level, long after formal teaching of spelling has ended, it appears that teach- 
ing rules is an inefficient pedagogical strategy. It may, however, be that in- 
dividuals take a period of time to internalize these rules and generate their 
own system of spelling strategies. Therefore, it may be most appropriate to 
provide learners with much exposure to correct spelling through reading 
and functional spelling activities and to the sounds of language through 
speech while encouraging them to discover conventional orthography and 
the rules that govern it through writing. 

Spelling ability appears to evolve in a predictable manner and, while 
some individuals will always have difficulty with spelling, most learn to 
spell very well. By the time they have reached university their rate of 
misspelled words has dropped from an average of about 13% in grade one to 
an average of about one half of 1% at university. 


Suggestions for Further Research 


In view of the current emphasis on “Whole Language,” attention should be 
given to determining the effect of this kind of language experience on spell- 
ing strategies. As well, the effect of diagnostic approaches to spelling where- 
in a student’s spelling strategies form the basis of instruction should be ex- 
amined. 
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The Influence of a Reading 
Treatment on the Written 
Compositions of Grade 11 Students: 
A Research Update 


The research reported below replicates on the grade 11 level our earlier study of the effect 
of a reading treatment on the reading achievement and writing proficiency of a sample of 
students in grades 9 and 10 (Belanger & Martin, 1984) and alters the factor we deemed 
responsible for the lack of writing improvement in the earlier study. In the current study 
writing was taught systematically and evaluated as part of students’ regular classroom 
work in English. Sixty-three experimental and 48 control students wrote both the pretest 
and posttest (Nelson-Denny Reading Test) and in-class essay test. Analysis of covariance 
indicates significant differences favoring the experimental group on both the reading test 
(p=0.046) and the writing sample (p=0.003) four months following the administration of 
the treatment. Final tests will be administered at the end of the following year, 16 months 
after the experiment began. 


Background 

Although correlations between reading achievement and writing proficiency 
are well known and well documented, a 50-year search for a reading treat- 
ment which improves writing skill has until now been unsuccessful. 
Moderate, statistically significant correlations have been reported between 
general reading and writing ability (Loban, 1976), reading and the syntactic 
complexity of writing (Hartman, 1984), reading ability and correct spelling 
and editorial usage (Pitts, 1985), reading and environmental factors 
(Weathermon, 1984), and the teaching of beginning reading and writing 
ability (Eckhoff, 1985). In a meta-analysis of 89 studies conducted between 
1950 and 1978 which correlated reading, writing, speaking, and listening 
measures, Hammill and McNutt (1980) reported that the median correla- 
tion between 37 measures of reading and writing was .61. However, because 
empirical studies have not been able to show a causative relationship be- 
tween reading and writing skills, these correlations have been poorly under- 
stood. 

Previous reading treatments have not improved writing ability. Treat- 
ments which have taught vocabulary development and paragraph reading 
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skills (Eurich, 1931), the SRA materials (Schneider, 1970), psycholinguistic 
methods of teaching remedial reading (O’Donnell, 1974), or stylistic and 
rhetorical principles (Austin, 1983) failed on the first step—the improve- 
ment of reading—and therefore conclusions could not be drawn about the 
hypothesized consequent growth in writing ability. An earlier study by the 
authors of this paper (Belanger & Martin, 1984) reported that a phonics- 
based reading treatment resulted in significant (p<0.0013) and substantial 
(about one half year’s growth on the test norms) differences on a stand- 
ardized reading measure but no differences on a writing sample. We conjec- 
tured that a lack of incentive for the students to write and a lack of 
adequate instruction in writing prevented the subjects from producing their 
best written work; experimental and control essays were uniformly poor. 

A small number of writing treatments which have taught aspects of the 
structure of English appear to have the potential of improving both reading 
and writing skill, but only studies using sentence combining on the elemen- 
tary level have produced completely unambiguous results. Sentence combin- 
ing studies by McAfee (1980, grade 5) and Mackie (1982, grade 4) resulted 
in significant improvements on both reading and writing measures. How- 
ever, a number of other studies of sentence combining at the elementary 
level have produced significant differences on some reading measures but 
not on others (Combs, 1977; Levine, 1976; Straw & Schreiner, 1982). On 
the grade 6 level, Kelley (1984) reported that teaching the structure of 
paragraphs and stories produced significant results on both reading and 
writing measures when compared with free reading, but other studies which 
have taught aspects of the structure of English have produced ambiguous 
results. Taylor and Beach (1984), Crowhurst (forthcoming), Braun and Gor- 
don (1984), and Bossone and Troyka (1976) reported significant changes in 
writing ability but inconsistent changes in reading ability as a result of the 
study of various schemata and text-structure treatments. (See Belanger, 
1987, for asummary of these and other studies.) 


Design and Procedures 
A true random pretest/posttest design with a midtest four months following 
the administration of the treatment was used in this experiment. True ran- 
domization was possible because teachers allowed groups of students to be 
withdrawn from each class while control students worked on individual as- 
signments. The design can be depicted as follows: 

R O1 X O03 O05 

R O2 .. 04 06 
Findings from the first four months (O3 and O4) are reported below. 

The 164 grade 11 students who wrote both the Nelson-Denny Reading 
Test (Form C) administered by the English Department in the fall and the 
department-wide Fall Essay Test were randomly designated experimental 
or control. In February, the treatment groups were administered the read- 
ing treatment, a four-page device designed to tap students’ phonic knowl- 
edge (knowledge of the code of English) to enable them to rely on it to de- 
code the printed page.! Five students, three experimental and two control, 
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who learned English as a second language after enrolling in grade one, and 
three experimental students who missed the treatment due to long-term ill- 
ness were eliminated from the experiment. In June, after eliminating school 
dropouts and transfers (25) and those missing either the essay (9) or the 
reading (19) tests, 111 students (63 experimental and 48 control) remained. 

The Nelson-Denny Reading Test, Forms C and D, were used as the read- 
ing measure in the experiment. Form C was administered by classroom 
teachers in September 1986 as part of regular English Department data col- 
lection procedures. Form D was administered in June by classroom teachers 
for the experiment. Twelve months following the administration of Form D, 
Form A will be used as a posttest. 

The writing sample consisted of two essays written during normal school 
testing periods as part of an assessment in English and marked by the class- 
room teachers during released time using procedures suggested by 
Diederich (1974). (School officials agreed to release teachers to score the es- 
says if substitutes were hired to teach the regular classes.) Papers were 
identified by a code number only, and examinations from the five classes 
were randomly intermingled prior to the marking sessions. Raters, there- 
fore, were unaware of the identities of the authors. One of the investigators 
(Belanger) joined the marking team. Teachers were accustomed to team 
marking because the department had, on occasion, taken a day to assess 
student writing as a team. In addition, four of the six evaluators had served 
together on provincial marking teams. Following Diederich’s suggestions, 
essays were assigned percentage scores by two primary raters. If these 
scores were nine marks or fewer apart, the paper’s grade was an average of 
the scores of the two primary raters. Reliabilities for the two primary raters 
(unadjusted scores) were .53 in the fall and .70 in the spring. If the scores of 
the primary raters differed by 10 marks or more, a third rater was assigned 
the paper to rerate without knowledge of the scores of the other two raters. 
Elimination of the discrepant grade resulted in interrater reliabilities of .85 
in the fall and .88 in the spring. Twelve months following the collection of 
the second writing sample, a third essay (written and evaluated under con- 
ditions similar to those used for the first two) will be used as a posttest. 

All statistical procedures were carried out on the UBC Amdahl V8 com- 
puter using the Statistical Package for Social Sciences Extended Version 
(SPSSX, Release 2.0). Two-way analysis of covariance (ANOVA) was 
selected to test major differences between the experimental and control 
groups because the pretest revealed differences (although nonsignificant) 
between the two groups on both the initial reading and writing tests (Table 
iy 

Interactions between the treatment and both the sex and the reading 
and writing achievement of the subjects were tested. To establish reading 
and writing achievement levels, students were divided into four categories 
based on their combined initial reading and writing scores. As Table 2 
shows, cell sizes for ability level ranged from 11 to 17 subjects. 
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Table 1 
Raw Scores by Test Period and Treatment 


a pe 
Fall Spring Difference 


Reading 


Experimental 63 62.67 22.1 70.73 22.4 8.06 
Control 48 58.90 18.6 62.44 21.9 3.54 
Exp. Girls 34 67.41 24.1 75.03 23.0 7.62 
Exp. Boys 29 57.10 18.4 65.69 20.9 8.59 
Con. Girls 24 62.21 215 65.88 25.6 3.67 
Con. Boys 24 55.58 15.0 59.00 172 3.42 
Essay 

Experimental 63 61.38 9.9 65.57 11.3 4.19 
Control 48 63.10 9.2 61.60 8.4 -1.50 
Exp. Girls 34 64.60 8.7 68.85 10.0 4.25 
Exp. Boys 29 57.59 9.9 61.72 11.8 4.13 
Con. Girls 24 63.83 8.2 64.08 9.4 0.25 
Con. Boys 24 62.38 10.3 59.13 6.4 -3.25 


Interrater reliabilities and correlations between reading and writing 
ability were calculated using PEARSON CORR (zero-order product moment 
correlations). 


Results 
Reading 
Analysis of the reading measure corroborates the findings of our earlier 
study (Belanger & Martin, 1984). Four months following the administration 
of the treatment, using the pretest as the covariate, the experimental 
groups were superior to the control groups at the .046 level of confidence on 
the June reading test (Table 3). 

As Table 1 indicates, the experimental and control students experienced 
substantial growth on the Nelson-Denny Reading Test over the four-month 
initial phase, but the experimental students (who gained 8.06 raw-score 
points or 12.9%) showed over double the gains of the control students (who 
gained 3.54 points or 6.0%). This difference represents about one half year 
on the test norms. This pattern is similar to that of our 1984 study which 
was conducted at the grade 9 and grade 10 levels and used the Davis Read- 
ing Test. 

Tests of interactions (Table 3) indicate that neither the sex of the subject 
nor the subject’s initial reading and writing ability influenced the results 
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Table 2 
Raw Scores by Test Period, Treatment, and Ability 


Fall Spring Difference 
n x s x s x 
Reading 
Low Exp. 17 42.00 7.2 53.00 14.4 11.00 
Con. 11 41.46 9.2 49.55 15.6 8.09 
2 Quart. Exp. 15 52.60 6.5 60.07 12.0 TcAT. 
Con. 13 51.85 5.8 56.54 9.8 4.69 
3 Quart. Exp. 14 62.21 8.5 70.57 11.4 8.36 
Con. 13 60.69 6.9 57.31 9.3 -3.38 
High Exp. 17 92.58 15.7 98.00 16.6 5.42 
Con. 11 82.55 20.8 88.36 28.0 5.81 
Essay 
Low Exp. 17 53.94 8.9 61.24 9.3 Teeie, 
Con. 11 53.00 5.7 55.09 6.4 2.09 
2 Quart. Exp. 15 59.07 6.9 59.33 8.2 0.26 
Con: 13 59.85 5.4 60.54 6.7 0.69 
3 Quart. Exp. 14 64.14 5.9 69.21 12.6 BIOs, 
Con. 13 68.54 morTs 63.23 B7 -5.31 
High Exp. 17 68.59 10.1 72.41 10.3 3.82 
Con. 11 70.64 5.5 67.46 10.3 -3.18 


significantly. As Table 1 shows, the experimental boys showed somewhat 
more growth (8.59 points) than the experimental girls (7.62 points), while 
the control girls made slightly greater gains than the control boys. There 
were also small differences when the groups were divided on the basis of ini- 
tial reading and writing ability. Table 2 reveals that all but the top quartile 
of experimental students showed reading gains greater than those of the 
control students. In our 1984 experiment, on the other hand, nonsignificant 
differences favored the girls and the high-ability students. 


Results 

Writing 

Analysis of the students’ essays revealed significant differences between the 
experimental and control groups, unlike the analysis in our 1984 project. As 
Table 4 shows, when the experimental and control groups are compared on 
the June writing sample using the pretest as covariate, the experimental 
students are superior to the control at the .003 level of confidence. Table 1 
reveals that the experimental students gained 5.69 points over the control 
students. The experimental students gained 4.19 points while the control 
groups lost an average of 1.5 points.” 
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Table 3 
Spring Reading Score Using Fall Reading Score as Covariate 


eee 


Source DF Mean Square F Prob 
Covariate 1 41181.28 325.21 0.0001 
Main Effects 1 527.87 4.08 0.046 
Interactions 
Sex 1 12.02 0.09 0.761 
Ability 3 180.05 1.39 0.250 
Explained 13 3301.87 25.51 0.0 
Residual 97 129.42 
Total 110 504.34 


As with the reading results, there were no significant interactions be- 
tween the treatment and the sex or the ability of the subjects (see Table 4). 
As Table 1 shows, the experimental boys and girls showed almost equal 
pre/post gains (4.13 and 4.25 points respectively). The control girls, on the 
other hand, showed only slight gains while the control boys accounted for all 
the losses shown by the control groups. When the groups are examined by 
ability (Table 2), all but the second quartile experimental students show 
substantial writing gains over the control students. Considering the stu- 
dents’ writing, then, the treatment did not appear to offer a particular ad- 
vantage to boys or girls or to weak or strong students. 


Discussion and Implications 
In 1984 we reported that significant (»<.0013) and substantial (almost one 
half year’s growth on reading-test norms) gains by experimental students 
on a standardized reading test were not accompanied by measurable 


Table 4 
Spring Essay Score Using Fall Essay Score as Covariate 


Source DF Mean Square F Prob 
Covariate 1 3912.96 59.73 0.0001 
Main Effects 1 576.56 9.14 0.003 
Interactions 
Sex 1 26.29 0.42 0.520 
Ability 3 119.76 1.90 0.135 
Explained 13 427.98 6.78 0.0001 
Residual 97 63.11 
Total 110 106.23 
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changes on writing measures. The changes on the writing measures were 
not, in fact, generally in the hypothesized direction. Measures taken both 
three and six months following the administration of the treatment showed 
that the major effect of the reading treatment was evident in the first three 
months of the experiment and that the gap between the experimental and 
control students continued to widen, albeit more slowly, during the second 
three months. The writing results, on the other hand, were random and er- 
ratic at both test periods. 

We felt in our 1984 study that a major difficulty with our findings was 
the lack of incentive to write provided by the data collection procedures. 
Students were asked to write an essay in October, February, and June dur- 
ing their regular class periods but without preparation, response, or evalua- 
tion. The essays were neither marked by their teachers nor returned. Al- 
though this type of demand assignment with its lack of preparation, au- 
dience, response, and publication has been one of the hallmarks of research 
on writing, there is little reason to expect that students will produce their 
best writing—writing which would accurately reflect their ability—under 
such conditions. Nor did writing appear to be a major part of the curriculum 
in the classrooms studied. As is the case in many English classes (Applebee, 
1981), writing was sometimes expected but seldom taught. 

Consequently, for the current experiment we sought a setting in which 
writing was taught systematically and valued as an important form of com- 
munication. In addition, the writing samples were required to be written as 
part of the students’ regularly assigned-classroom work, marked by teams 
of teachers, and counted as a portion of the students’ marks in English 
class. Teachers in the department agreed to assign essays as term tests in 
writing, administer them under examination conditions, and score them on 
a day released from school. There were also three minor differences be- 
tween the current experiment and the 1984 experiment, differences which 
were not expected to influence the findings: the grade level (9 and 10 in 
1984; 11 in the current experiment); the size of the experimental population 
(198 then; 111 now); and the standardized reading test (Davis in 1984; Nel- 
son-Denny currently’). 

We believe these results are important because they answer a question 
that has intrigued researchers for at least half a century: Does an improve- 
ment in reading ability bring about improvement in writing ability? Pre- 
vious research has been able to demonstrate only a correlational rela- 
tionship between reading treatments and writing achievement, a relation- 
ship which could be attributed to a variety of causes. The current study, on 
the other hand, shows that a reading treatment has a direct influence on 
writing ability by establishing a causal link between the two, a link which 
suggests an explanation for the vast amount of correlational data in earlier 
research. As such, the findings shed new light on the theoretical bases of 
reading and writing. 

The major question raised by the findings is, What characteristics of the 
treatment are responsible for the observed changes on the reading and writ- 
ing measures? As our outline of the conceptual basis of the reading treat- 
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ment (Note 1) suggests, we attribute the results of the study to the stu- 
dents’ improved automatic handling of the code of written English. 

. Obviously this is speculative inference from the data, not direct observa- 
tion. But because the results offer little comfort to prevailing theory in the 
field of reading, some explanation is in order as to why we undertook this 
study and its predecessors and what theoretical implications we see as ac- 
counting for results that seem to challenge conventional wisdom. What fol- 
lows is our interpretation of the results, along with the direction in which 
they seem to point. 

On the reading measure, gains resulted, we believe, from the students’ 
improved ability to decode written words automatically and fluently. The 
treatment forced students to rely on their ability to decode the sequence of 
sounds embodied in the written word, an ability students have apparently 
acquired through nine or more years of exposure to print in the school sys- 
tem and elsewhere. 

Much current reading instruction (such as word guessing from context), 
however well intentioned, may have the effect of interfering with induced 
capacity by introducing a conscious element in a process best carried out 
automatically. It is this interference which we believe is counteracted by the 
SOS (Sequence of Sounds) technique, and if the interference from coun- 
terproductive instruction is removed, preexisting but unused capacity can 
manifest itself. 

The effect on writing, on the other hand, while also a direct result of im- 
proved ability to handle the written code of English, is probably more subtle 
and based on a variety of benefits produced by improved (or newly lib- 
erated) facility in automatic decoding during reading. Changes on the writ- 
ing measures are likely the cumulative effects of a number of separate 
aspects of handling the written code, each of which might be enhanced by 
formal instruction, with improved reading ability providing the foundation 
on which such instruction might prosper. 

On the word level, improved handling of the written code may lead to in- 
creased facility with English spelling and, in due course, to broader 
vocabulary. Students taught to guess or memorize words when they are 
reading are likely to rely on guesswork and memory in their writing. On the 
other hand, students who have cracked the code and who become proficient 
in using the English spelling system can rely on the code for the many 
regular English words, thus reducing the memory load to those words, or 
word portions, which do not fit the system. This, of course, is not a change 
that comes overnight, but one which takes place gradually as writers figure 
out the system; nonetheless, small improvements might be expected in the 
beginning, growth which, when taken cumulatively with other develop- 
ments, results in significant overall improvement in writing ability. 

Vocabulary is, we believe, amenable to similar kinds of enrichment. Stu- 
dents who have cracked the code and learned to “hear” words accurately as 
they read them—rather than to guess what the word might be—know when 
they have encountered an unfamiliar word. Having perhaps heard the word 
once or twice, they can decide whether they want to add it to their 
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vocabularies. However, readers who rely on guessing rather than the code 
have two variables to try to control: first, whether they have guessed cor- 
rectly what the word is (that is, if they have guessed correctly what spoken 
word has been printed), and second, whether (now having guessed what the 
word is) they can proceed to guess what it means. For example, in the sen- 
tence “the boy was adamant about his role in the theft of the candy,” the 
decoder hears the sequence of sounds and knows whether “adamant” is 
part of his vocabulary. The word guesser, on the other hand, does not know 
whether the word is adamant, abhorrent, aberrant, or what, and therefore 
reads “the boy was something about his role....” Unless he returns to 
adamant to decode it (or asks his teacher what the word is) he cannot know 
whether the word adamant is in his vocabulary because he has not read 
adamant but merely glossed over it or guessed what kind of word might fit 
in adamant’s position. If he has not read the word, he does not know if he 
knows it; if he does not know whether he knows it, he has no chance of 
either understanding or learning it. 

On the sentence level, automatic decoding takes the emphasis off the 
word and puts it on the phrase, clause, or sentence. This gives the student 
the opportunity to observe and absorb the cadence and rhythm of written 
English, as well as the sense supplied by syntax that is different from the 
sum of the particular word meanings. The word guesser, on the other hand, 
skipping some words completely and returning to others to puzzle them out, 
cannot hear the ebb and flow of the language, and the strength of the syn- 
tax of English will not be absorbed. Reformed word guessers who have been 
taught to rely on the code, on the other hand, can benefit from listening to 
the rhythms of the page, and can slowly begin to develop a sense of the 
metrics and logic of written English. But especially beyond the sentence, 
there will be little flow of meaning for the unreformed word guesser who 
must read from guess to guess, with little time for the larger issues in the 
text. 

One of the enduring theoretical bases for expecting relationships be- 
tween reading and writing has been the observation that reading provides 
ideas and models for writing. Under this assumption, one who reads widely 
gains a command of things to write about and a notion of how writers or- 
ganize, provide evidence, and nurture readers. Even if improved reading did 
not automatically lead to more reading, it would increase what students 
learn from the reading they do. As with spelling, vocabulary, and syntax 
changes discussed above, improvement in ideas and style probably con- 
tributed some measure to the significant improvement in writing ability 
noted in the study. 

Similar processes appear to be operating on the level of standard 
editorial usage. Readers who decode automatically and hear the rhythm and 
flow of the language are likely to hear the need for punctuation. On the 
other hand, readers who concentrate on guessing what words might fit into 
a given sentence slot are likely too preoccupied with error reduction to have 
much time to sort out punctuation. 
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We believe that all of the above contribute in some way to the improved 
writing which was brought about by the foundation laid by the reading 
treatment. While some of these changes in writing might be expected to be 
automatic, it seems more likely, especially in light of our 1984 results, that 
they can be enhanced by instruction. Our findings suggest that automatic 
decoding is basic to such instruction. 

The reading results simply confirm findings of a number of earlier ex- 
periments using the reading treatment (Belanger & Martin, 1984; Martin, 
1975a, 1975b). Taken by themselves, these results have implications for 
both schools and research. First, we should question the current complacen- 
cy about the reading competence of average and above average secondary 
students. Both categories of students show rapid and substantial improve- 
ment in reading in this study, suggesting that learning to rely on their 
knowledge of the code of written English is not the exclusive domain of 
those traditionally stigmatized as remedial. The implication is, of course, 
that even the most able readers are in some way impaired readers. We 
believe, therefore, that educators should consider applying the technique 
across the board to secondary students, not just to those who obviously pose 
a problem.* 

Second, the reading results raise questions about the fundamental basis 
of currently accepted reading theory. By its very nature, the reading treat- 
ment runs athwart much of this accepted theory including the orthodox 
tradition from Gray, Gates, Harris, Robinson, and others and the more 
recent uncertainty-reduction views of Goodman, Smith, and like-minded 
theorists. Similarly challenged are secondary reading assumptions leading 
to techniques that add to the reader’s self-consciousness about reading, 
which in turn may diminish the capacity for automatic decoding. 

The study also raises for future research these questions: 


1. What kind of instruction in writing is most productive following reading 
improvement (e.g., sentence structure, voice, coherence, ideas, 
vocabulary, editorial usage, or spelling)? Is the grade level of the sub- 
jects a factor in the effectiveness of the instruction offered? 

2. Is the mode of writing a factor? This experiment focused on the ex- 
pository essay because the grade 11 teachers in the study traditionally 
taught the expository essay as a major part of their curriculum. The ex- 
pository essay may or may not be the most suitable mode to teach to 
capitalize on reading improvement. 

3. What are the long-term effects of this reading treatment on writing 
ability—after five or six years, for example? 

4. Ifa reading treatment that explicitly ignores and bypasses instruction 
in meaning can improve comprehension and enrich the semantic powers 
of students in their writing, what implications does this have for mean- 
ing-based reading theory? 
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1. 


Notes 


The references cited below are essential to even a minimal understanding of the SOS 
procedure and rationale. But the essence of the rationale is suggested in this passage by 
the inventor of the technique: 

All the SOS does is to remove from the conscious mind any consciousness in 

the activity of reading, except the consciousness of the messages presented in 

print, exactly as in hearing. Then the mind can think about the thoughts 

presented while reading. Thus the technique, far from teaching people how to 

read, stops them from not reading, and allows the subconscious to do what it 

has been able to do since junior high school, or at least the ninth grade, trans- 

late code into sound into word into sentence into paragraph and so on, while 

the reader can think about the information he is receiving. (Martin, 1976, p. 

12) 
The procedure involves having subjects read aloud first some single nonsense syllables, 
followed by polysyllabic units, followed by a mixture of simple words and nonsense syll- 
ables, all pronounced accurately without being read aloud by the administrator, followed 
by short, regular English words which are (under suitable incentives) read much too fast 
to permit total accuracy, so that conscious control is relinquished and subconscious com- 
petence can assert itself. This is preceded by a paragraph of ordinary silent reading and 
followed by the next paragraph in the same material to demonstrate to the reader that a 
change has occurred. The procedure is administered optimally to a group of four people, 
once only, in less than an hour, but can be done successfully with an individual or a group 
smaller than four. Additional and indispensable aspects of procedure, along with their 
theoretical bases, are described in Martin (1976, especially chap. II, pp. 22-33) and more 
casually in Martin (1977). Belanger (1978, pp. 250-53) offers a brief description of the 
treatment. 
Beginning reading programs which share a similar theoretical orientation include the Lip- 
pincott series and the DISTAR program. 
The two essay tests were not evaluated simultaneously; the fall essays were evaluated in 
the fall and returned to the students, and the spring essays were marked in June. We 
suspect that the raters were more demanding in the spring (after they had taught the 
essay form) than they were in the fall. This might account for the apparent losses by the 
control groups on the second set of essays. The reading tests, on the other hand, were 
parallel forms of equal difficulty; consequently, increasing demands of raters would not be 
a factor in the grades. 
The Davis Reading Test was selected for the 1984 project in part because the school 
district used it as a system-wide test in grade 9. However, we discovered that by the end 
of grade 10 many students achieved almost perfect scores on the test. Therefore, we 
decided that the ceiling on the Davis test would be too low for grades 11 and 12 and chose 
the Nelson-Denny Reading Test which appeared to be more discriminating for 
upper-ability students. 
The nature of the technique limits expenditure to training of staff and about one hour of 
time for each group of four students in a single, once-only administration. (No commercial 
materials are required.) Teacher/administrator time and minor adjustments to class 
schedules are needed to make this possible. 
A problem not addressed in this study is that not all willing teachers may be able to give 
the technique because of its exacting demands on the administrator. Two to three days of 
instruction in its use appear necessary, along with an open mind and the making of tapes 
of sample administrations for suitable feedback to prospective administrators of the tech- 
nique. 
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The Topic Selections 
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Adult Student Writers 


In a one-year study, 249 adult writers of the Alberta Universities’ Writing Competence Test 
were found to differ from younger writers in their performance and topic selections. Older 
writers performed markedly better than younger ones in terms of grade received. While 
different topics appealed more to adult writers than to younger writers, there was no cor- 
relation for either group between the popularity of a topic and the success of those choos- 
ing it. The study concluded that writing topics need not be specifically targeted toward 
adult writers, nor need personal topics be avoided, as these writers wrote with comparative 
success across a wide range of topics. 


Introduction 
Adult students comprise a growing proportion of college and university stu- 
dent populations throughout Canada. At the University of Lethbridge, for 
example, students aged 25 and over made up 28.5% of the full-time student 
population in the fall of 1987 (University of Lethbridge, 1988) compared 
with 16.3% in the fall of 1972 (University of Lethbridge, 1973). When the 
large numbers of adult part-time students are also counted, it becomes clear 
that adult learners form a “new majority” of university student populations 
(Campbell, 1984); administrators and teachers in postsecondary institutions 
can no longer assume that their students will have the academic experience 
‘and preparation of recent high school graduates. In fact, evaluation of stu- 
dent performance becomes more problematic when students vary greatly in 
academic background. A particularly difficult problem in evaluation is a test 
of writing competence—another increasingly common phenomenon in uni- 
versities and colleges. 

The writing of adults, in terms of performance and composition process, 
has been studied to only a limited degree and primarily for purposes of con- 
trast with younger writers (Odell, Cooper, & Courts, 1978). Such studies 
usually define adults as older teenagers or college students. In examining 
adult attitudes toward writing, Connors (1982) found that adult students 
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lacked confidence as writers, spent more time revising essays, and asked for 
more direction and limits on writing assignments than younger students. 
Hayes and Flower (1986) studied the planning process of older students and 
adults, a process characterized more by the use of problem solving strat- 
egies than by the use of known writing formats. Shook (1985) argued 
against setting “personal” writing topics for adults, for whom the self is “no 
longer the only real source of information or even of interest” (p. 205). Such 
studies raise questions about topic design, particularly on writing exams 
where a wide age range of students can be expected; yet it appears that no 
one has examined comparatively the patterns of topic selection and perfor- 
mance of adult and younger writers. 

In 1982 Hoetker commented, “Research to date on topic effects is incon- 
clusive” (p. 378), and in key areas of topic research this is still true. On a 
Teaching Certificate exam with multiple writing topics, the popularity of an 
individual topic has been found to be unrelated to the mean scores for that 
topic (Brossell, 1986). In contrast, Gee (1987), working with grade 12 
exams, found that popularity varied inversely with score. Carlman (1986), 
also with grade 12s, discovered no significant differences between mean 
scores on topics within a single “mode” (e.g., argument, narration), but did 
note significant performance differences between modes. A study of holistic 
essay marking (high school level) found that an argumentative topic led toa 
greater variety of responses than did a descriptive one and hence perhaps to 
a greater spread in holistic marks received (Nagy, Evans, & Robinson, in 
press). Researchers have found the subject matter of questions not to be a 
variable in score received by university sophomores (Brossell & Ash, 1984); 
others suggest that subject matter, as it relates to topic-specific background 
knowledge, is a factor in performance (for grade 10 students, Langer, 
1983). The delicate question of writers’ emotional responses to topic and 
subject matter has been investigated by Brand and House (1987), who con- 
cluded that topic does affect the “emotional fluctuations” and performance 
of (college) student writers. 

The purpose of this study is to examine the topic selections and perfor- 
mance of adult writers (aged 25 and over) of the Alberta Universities’ Writ- 
ing Competence Test, administered since 1983 at the four universities in 
the province. The AUWCT is a compulsory, timed (two-hour) test in which 
students respond to one of four questions of general interest in an ex- 
pository or argumentative essay of about 400 words, expository questions 
being slightly more common. The four possible grades are Satisfactory and 
Marginally Satisfactory (passing grades), Marginally Unsatisfactory and 
Unsatisfactory (failing grades); the marking is detailed rather than holistic, 
that is, of the “error count” kind. 

Our questions are the following: How did the older writers fare on the 
AUWCT, and how did their performance compare with that of younger 
ones? What topics appealed or did not appeal to adults and to youth? And 
finally, did topic choice seem to make any difference in performance? 
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Methodology 

This study draws on the AUWCT exams written at the University of 
Lethbridge during a one-year period (July 1986 to June 1987) comprised of 
a total of 1,175 exams written on ten dates. From this total we excluded 
papers identified by the markers as “English as a Second Language” papers 
(147 exams). The remaining papers were divided into “youth” exams writ- 
ten by students aged 20 and under (706 papers), “young adult” exams writ- 
ten by students aged 21-24 (73 papers), and “adult” exams written by 
students aged 25 and over (249 papers). Our research sample of 249 adult 
writers had a mean age of 32.4; an age profile of these writers appears in 
Figure 1. 

Our data were gathered from the exams themselves, which were stock- 
piled at the university along with the marking sheets. We recorded the 
marks achieved (S, MS, MU, or U) by adult and youth writers over the year; 
for purposes of greatest contrast we discounted the 73 young adult papers. 
We went on to record the question selections of adult writers on the 10 
dates, that is, how many adults chose each question on each date. Using chi- 


25-29 30-34 35-39 40-44 45-49 50-54 


Figure 1: Age profile of adult writers (numbers are the actual numbers of students in 
each age range) 
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square analysis of adult question selections among the four questions on 
each exam, we found that on five of the 10 dates their selection among the 
four could not be attributed to random distribution (four dates significant at 
the 1% level, one at the 5% level). These five exam sittings accounted for 
60% of the adult writers and were in fact the five largest of the 10 exam sit- 
tings. 

For these five target dates we contrasted the adult pattern of question 
choice with that of the youth (writers 20 years and under) by recording the 
results both numerically and graphically. We also analyzed the relationship 
between popularity of a question and the success of those choosing it (suc- 
cess determined by pass rate) for both the adult and youth samples by per- 
forming a linear least squares analysis as well as higher order correlations. 
Again, the young adult population was not included in these analyses; their 
numbers were too small to provide reliable statistics. 


Results 

The profile of performance by grade received indicates that over the year 
the adult writers achieved a pass rate of 73.9% compared with the youth 
pass rate of 61.8%. We illustrate performance by grades in Figure 2 (for 
each grade category the left-hand bar represents the percentage of adults 
achieving the grade and the right-hand bar represents the percentage of 
youth). 

With the knowledge that the adult writers were markedly more success- 
ful than the youth on these exams, we turned our attention to their topic 
choices. We determined that question selections were fairly evenly distrib- 


uted among the four questions of the sets over the year, both for adult 
Y rai) | 


S MS MU U 


Figure 2; Grade frequencies for adult and youth writers over a one-year period 
(bars represent % achieving a mark) 
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writers and for youth; that is, writers did not particularly favor questions 
appearing first, for example, in a set. From the target dates we had 20 
topics to consider (see Appendix 1 for the actual questions) and a sample of 
writers aged under 21 with which to contrast the adults. The figures below 
illustrate contrastive topic selection on these five exams. Each bar repre- 
sents the percentage of students who chose a particular question; each set 
of four bars totals 100%. 

From these figures it is apparent that some topics were popular with 
both adults and youth, and there were a few which were universally un- 
popular. But there were also topics which had different appeal for the two 
age groups. 

Three topics had obvious attraction for both age groups: seat belt laws, 
the value of fashion, and the choice of a city in which to live. These three 
were among the top five choices on both lists selected by more than 40% of 
both adults and youth. The two groups also agreed on four of the bottom 
five choices, topics chosen by fewer than 10% of those writing. The rejected 
four were freedom of speech at universities, international terrorism, stu- 
dents’ union issues, and world trust and understanding. However, in other 
cases the two groups disagreed sharply on the attractiveness of a topic. 

Some topics appealed much more to adults than to youth: the charac- 
teristics of an ideal marriage partner, the powers of unions, government 
support for family farms, and modern improvements in daily life. And the 
young writers responded to some topics which did not interest adults: a 
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Figure 3: July 24, 1986: Adult and youth question selection 
Relative popularity of individual questions with adult (hatched) and youth (clear) 


writers (bars represent % choosing a question) 
Topics: 1. favorite sport 2. ideal marriage partner 3. freedom of speech 4. terrorism 
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Figure 4: August 6, 1986: Adult and youth question selection 


Relative popularity of individual questions with adult (hatched) and youth (clear) 
writers (bars represent % choosing a question) 


Topics: 1. today’s youth 2. winning in sports 3. trust among nations 4. 20th-century 
daily life 
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Figure 5: September 5, 1986: Adult and youth question selection 


Relative popularity of individual questions with adult (hatched) and youth (clear) 
writers (bars represent % choosing a question) 


Topics: 1. a second language to study 2. education 3. seat belt laws 4. time capsule 
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Figure 6: November 29, 1986; Adult and youth question selection 

Relative popularity of individual questions with adult (hatched) and youth (clear) 
writers (bars represent % choosing a question) 

Topics: 1. students’ union issues 2. fashion 3. humor 4. labor unions 
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Figure 7: April 6, 1986; Adult and youth question selection 

Relative popularity of individual questions with adult (hatched) and youth (clear) 
writers (bars represent % choosing a question) 

Topics: 1. university instructors 2. city to live in 3. family farms 4, what work 
teaches us 
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favorite sport or game, the importance of winning in sports, and the teach- 
ing abilities of university instructors. But the popularity of a question and 
the success (as measured by passing the exam) of those who chose it were 
not correlated. 

The linear least squares analysis to determine the relationship between 
popularity of a question and the success of the students choosing it revealed 
no correlation whatsoever, either for adults (Figure 8) or for youth (Figure 
9). The scatter diagrams in the figures illustrate this graphically. The slope 
of the regression line for adults was .073, for youth .056; clearly, neither 
adults nor youth in this sample fared any better or worse on unpopular or 
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Figure 8: A scatter diagram relating % achieving a passing grade to the % choosing 


a given question for adult writers (the dashed line represents the baseline pass rate 
for this group) 


54 


Topic Selections and Performance of Adult Writers 


100 


79 


s10) 


Percentage Achieving a Passing Grade 


29 


0 20 20 79 100 


Percentage of Youth Choosing a Question 


Figure 9: A scatter diagram relating % achieving a passing grade to the % choosing 
a given question for youth writers (the dashed line represents the baseline pass rate 
for this group) 


popular questions. Three higher order correlations were also tested unsuc- 
cessfully. 


Discussion 
Most questions on the AUWCT are closest to the “moderate information 
load” described by Brossell (1983) in his discussion of rhetorical specifica- 
tion in essay examination topics. The AUWCT questions are more than 
“briefly stated subjects” yet have less than “a full rhetorical context.” They 
may elicit either “reflexive” or “extensive” responses (Emig, 1971) and oc- 
casionally impassioned direct address. The official guidelines require that 
questions be of general interest, raise only one issue, and avoid highly emo- 
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tional issues. On each paper, of four questions, three are expected to be 
answerable by non-Albertans and at least one to be of particular interest to 
students from rural backgrounds, from nonacademic backgrounds, and 
from outside Canada. 

The actual questions are painstakingly written, evaluated, discussed and 
selected—months in advance—by individual instructors, departments at the 
four provincial universities, and the interuniversity committee which over- 
sees the exam and makes the final topic selections. Still, students (and 
sometimes markers) commonly complain that they “didn’t like any of the 
questions.” Students also assert that they “didn’t know anything about any 
of the topics,” which no doubt is sometimes true. The instructions on the 
exam disclaim the need for any “specific knowledge” of a topic, but that 
may be of little comfort to the exam writers. 

It is apparent that in this sample, adult writers performed more success- 
fully than youth in their responses. These adults are not the “basic writers” 
often discussed in adult composition research, and clearly it would be a mis- 
take to assume that adult students’ writing will be less proficient than that 
of younger students. But these older writers differ from their juniors not 
only in performance but also in topic selections, which we can only assume 
is deliberate and reflects to some extent personal interests and subject 
preferences. What, then, can we infer from their topic choices? 

First, we note a tendency on the part of the adult writers to choose topics 
on which they can bring to bear rather broad experience in life and work: 
the powers of labor unions, the qualities of an ideal marriage partner, and 
assessment of today’s youth. Youth, by contrast, tended to favor topics of 
more immediate and less philosophical interest: sport, fashion, or seat belt 
legislation. Shook (1985) advocates that we avoid setting personal, intro- 
spective topics for adults. But the adults in our study by no means shunned 
such topics; in fact they seem to have sought them out, given that they 
chose from a set of four topics which could elicit either reflexive or extensive 
responses. 

In selecting a particular topic, we suggest that these adults (and these 
youth) chose topics about which they felt authorized or qualified to write. 
Hoetker and Brossell (1986) refer to “content-fair” topics as “sets of topics 
that provide the great majority of examinees with the chance to write on a 
subject about which they know a good deal” (p. 329). All the topics used in 
the AUWCT evaluations seem in this sense content-fair, but apparently the 
older writers felt better qualified to write on certain topics than on others. 
Those that they chose seem to draw less on knowledge about current events 
or topical issues than on experience of life, and it is here, we suspect, that 
these adults found their sense of authority to write. 

It is interesting to note that for neither group was performance on a 
topic correlated with its popularity, thus confirming the results of Brossell 
(1986) and contrasting with the findings of Gee (1985). Gee, finding the fre- 
quency of selection of a topic inversely proportional to the score received, 
suggests that markers grow bored and lethargic marking the same topic 
over and over, then “perk up for those rare few on a different topic” (p. 84) 
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and that marked interest or lack of it may result in higher or lower grades. 
However, in this case it may be that what is a majority choice for adults is a 
minority choice overall and perhaps of more appeal to jaded markers. 


Conclusions 
The special needs, interests, and abilities of the adult students who con- 
stitute an increasingly large proportion of university and college student 
populations require the consideration of these institutions. This study and 
others indicate that many adult students, although often formally un- 
prepared, bring to their studies high levels of readiness, maturity, and com- 
munication skills that make them in many ways “advantaged” students. 

This study has several implications for those teaching composition to 
adult students or designing instruments of writing assessment. First, it 
seems clear that one need not choose topics targeted at a particular age 
group’s experience and interests, despite the tendency of the adult writers 
to select topics in line with their more extensive life experiences; the adult 
writers in this study chose a broad range of topics, adapting their responses 
to their own experience and range of knowledge, and succeeded very well. 
Second, one need not avoid setting topics which are personal or introspec- 
tive so long as the writers are not compelled to choose them. Given a choice, 
our adult writers often chose to write with the authority of personal ex- 
perience and sometimes with what we sensed to be a pleasant introspection, 
a vein of reflection generally missing in the papers of the younger writers. 
Finally we can only conclude that, in general, age is no disadvantage to 
these adults as writers; markers often comment appreciatively on the “ma- 
ture voice” behind adult writing, however unprepared or unpracticed the 
writers. 

Research concerning writing topics has focused primarily on topic for- 
mat and effect on writing performance. We suggest that a shift in attention 
be made from topic format to the writer. What factors make a topic both ac- 
cessible and appealing to writers of different ages? A greater focus on 
writers’ perceptions of a topic and motivations for choosing it would be a 
useful addition to current research in topic analysis. 

Finally, there seems to be a gap in composition research in the area of 
adult writing skills, with the exception of adult basic education. Adults now 
appear in all our university, college, and community composition courses 
and workshops; we need to know more about how they write, why they 
write as they do, and how best to help them improve their writing. For years 
composition research has concentrated largely on the writing of children 
and youth; attention to the writing of adults is long overdue. 
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Appendix 1: AUWCT Questions from Target Dates 
July 24, 1986 


1. What is your favorite sport or game, and what makes it enjoyable? 

2. What are the characteristics of the ideal marriage partner? Support 
your conclusions. 

3. Recently, South African Ambassador Glen Babb was prohibited from en- 
gaging in a debate at the University of Toronto Law School. Opponents 
of his visit argued that Babb would be defending an indefensible policy: 
apartheid. Other students argued that the refusal to allow Babb to par- 
ticipate in a public debate was a violation of a basic human right: 
freedom of speech. 
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What restrictions, if any, should universities place on the selection of 
speakers for open debate? Support your conclusions. 

What steps do you think should be taken to help reduce the threat of ter- 
rorism in the world today? Support your conclusions. 


August 6, 1986 


1. 


Some older people say that young people today are generally self- 
centered and greedy. They argue that young people don’t care about the 
welfare of others or about causes that don’t concern them; they want 
only to obtain high-paying jobs and a luxurious way of life. 

Do you agree or disagree with this assessment of today’s youth? Support 
your conclusions. 


. “Winning isn’t everything. It’s the only thing.”—Former football coach 


Vince Lombardi 
Do you think that winning is the primary objective in sports? Support 
your conclusions. 


. Many people who are campaigning for nuclear disarmament argue that 


ways of promoting understanding and trust among nations must first be 
found; only then will disarmament be possible. 

By what specific means could we promote understanding and trust 
among nations? Support your conclusions. 


. What person, policy, or invention do you think has made a significant 


improvement in our daily lives in this century? Support your conclusions. 


September 5, 1986 


1. 


2. 


If you could choose one language other than your own to study, which 
would you choose and why? Support your conclusions. 

What are the qualities of a well-educated person? Support your con- 
clusions. 


. Some people argue that laws which make the use of seat belts compul- 


sory curtail the freedom of the individual; others, however, maintain 
that such laws are necessary. 

Should the wearing of seat belts in automobiles be compulsory? Support 
your conclusions. 


. The time capsule is a wonderful notion. The idea is to collect contem- 


porary artifacts in a capsule and bury it so that future generations—or 
perhaps visitors from outer space—will know something about the way 
we live. —adapted from Richard Cohen, The Washington Post 

What item or items would you put in a time capsule to characterize your 
own country as it is in 1986? Support your conclusions. 


November 29, 1986 


L 


2. 


If you were President of the Students’ Union at your university, what 
issue or issues would you devote your time to? Support your conclusions. 
Most of us spend far too much time and money keeping up with 
fashions. Why do we go to all this trouble? We should do away with 


59 


J. O’Shea and K. Beaty Chiste 


fashions in dress and design an inexpensive, comfortable garment for 
everyone, men and women, to wear. —Thetis Oliver 
Do you agree or disagree with this suggestion? Support your conclusions. 


. Atype of humour prevalent in our society is based on stereotyping, the 


attributing of characteristics to groups of people on the basis of race, 
sex, or nationality. For example, ethnic jokes and comic strips featuring 
“typical housewives” are both very popular. Although some people see 
no harm in this type of humour and may even see it as healthy, others 
think that it is offensive and that it reinforces harmful prejudices. 

Do you see any harm in this type of humor? Support your conclusions. 


. Most people agree that when the labour movement first began, powerful 


unions were needed to protect workers against appalling conditions and 
exploitation that verged on slavery. However, some say that today’s 
unions are too powerful: they argue, for example, that unions strike far 
too often, sometimes even to the detriment of the union workers them- 
selves. 

Do you think that the powers of today’s unions should be limited? Sup- 
port your conclusions. 


April 6, 1987 


1. 
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Although few university students complain about their instructors’ 
knowledge, they often complain about their instructors’ abilities to im- 
part that knowledge. Some have suggested that the problem would be 
solved if all university instructors were trained as teachers. 

Should all university instructors be trained as teachers? Support your 
conclusions. 


. If you had the opportunity to live in any city in the world, which would 


you choose and why? Support your conclusions. 


. Because of the difficulties of the last few years, many small family 


farms are on the brink of financial disaster, and many people believe 
that the government should assist these operations. However, farming 
is becoming more complex and expensive, and it seems likely that soon 
only the large corporate-owned farm (agribusiness) will be able to profit. 
Does the family farm deserve support from the government? Support 
your conclusions. 


. “T don’t like work—no man does—but I like what is in work—the 


chance to find yourself.” —J. Conrad 
What can work teach us about ourselves? Support your conclusions. 
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Promoting Moral Development 
Through a High School 
English Curriculum 


This study examines the effects of a senior high school English curriculum in stimulating 
moral reasoning development as defined by Lawrence Kohlberg. The specific hypothesis 
tested in a pretest/posttest control group design involving 44 grade 10 senior high school 
students was that an advanced level English literature curriculum focusing on moral is- 
sues would promote moral stage change. The major finding of the study was that the cur- 
riculum proved effective to a significant degree in stimulating the growth of moral 
reasoning in preconventional students, but when the performance of the experimental 
group was compared with that of the control group, there was no overall statistical sig- 
nificance. The discussion section recommends psychological goals, beyond moral reason- 
ing development as defined by Kohlberg, as appropriate for an English curriculum. 


Introduction 

Much research evidence supports the hypothesis that the developmental 
level of moral reasoning, as defined by Kohlberg’s (1981) cognitive-develop- 
mental theory of moral development, can be advanced through group dis- 
cussion of moral dilemmas and through exposure to reasoning at a higher 
stage. Academic subjects provide ample opportunity for discussion of ethical 
and social issues, yet there appears to be no research published document- 
ing the impact of an English literature curriculum which has as an explicit 
goal the raising of moral reasoning level. This paper first offers a brief 
review of the literature on classroom moral stage change, and second it 
describes the effects on moral stage of a pilot curriculum in English litera- 
ture among grade 10 academic students in New Brunswick. 

The investigator’s hypothesis was that the use of literary material of 
high caliber and a pedagogy that emphasizes discussion, exploration, and 
debate of issues in the lives of literary characters, characters in film, and in 
the students’ own lives—all in a supportive environment—will stimulate 
the moral reasoning capabilities of the students as they are encouraged to 
confront the greater adequacy of moral reasoning at a stage above their 
own. 
The investigator recognizes that Kohlberg’s theory is cognitive in its em- 
phasis and that there is more to moral development than his ascending 
stages in conceptions of justice. Nevertheless, Kohlberg offers the re- 
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searcher a well-tested and validated instrument for the assessment of a 
critical element in moral development. 


Review of the Literature 

Moral judgment development, according to Kohlberg (1984), proceeds in a 
step-like manner. Each step or structure in his moral reasoning scale is a 
qualitative reorganization of a person’s pattern of thought that integrates 
and subsumes the insights achieved at lower steps; each stage represents a 
more adequate and logical way of understanding the moral world than the 
previous one. The stages are divided into three levels, each containing two 
stages. At the preconventional level (Stages 1 and 2), characteristic of chil- 
dren and developmentally slow adolescents and adults, the individual is 

responsive to cultural rules and labels of good or bad, right or wrong, but interprets 

these labels either in terms of the physical or the hedonistic consequences of action 


(punishment, reward, exchange of favors) or in terms of the physical power of those 
who enunciate the rules and labels. (Scharf, 1978, p. 308) 


At the conventional level (Stages 3 and 4), the level of the average adoles- 
cent and adult, the individual understands and tries to uphold the values of 
his or her society; the expectations of others—peers and authorities—have 
been internalized. At the postconventional level (Stages 5 and 6), charac- 
teristic of very few adults, “customs and social rules are critically examined 
in terms of universal human rights and duties and universal moral prin- 
ciples” (Jennings, Kilkenny, & Kohlberg, 1983, p. 282). Table 1 clarifies at 
each stage what is considered right. 

In the last 15 years, educational research influenced by Kohlberg’s the- 
ory of moral development has followed two main lines: (a) interventions in 
classrooms having the explicit goal of raising the stage of moral reasoning 
among members of the experimental group, and (b) the creation of demo- 
cratic high schools, where the researcher has looked specifically at the inter- 
action of the stage of moral atmosphere, the norms of the group, and the 
individual’s structures of moral reasoning. The first line is indebted to 
Piaget and Inhelder’s (1969) and Kohlberg’s (1981) interactive models of 
cognitive and moral development; the second line is more sociological, ex- 
amining the impact of shared norms on individual moral growth. This paper 
considers the research on classroom intervention. 

There have been many studies to validate the effect of classroom moral 
discussions on moral reasoning growth. Kohlberg’s hypothetical moral 
dilemmas, originally a tool of assessment, were first used by Blatt and 
Kohlberg (1975) as a “curriculum” in an attempt to demonstrate that sig- 
nificant growth in moral reasoning, as measured by the Moral Judgment In- 
terview (MJI), could be promoted in a group exposed to discussion of moral 
dilemmas, in contrast to a control group not similarly exposed. This study 
and others by researchers such as Schaffer (1974) served as pilot projects 
for this form of moral education. Typically, a moral dilemma is read aloud 
by the teacher and followed by class discussion. A mix of students at dif- 
ferent stages of moral reasoning has been found preferable because a 
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Table 1 
Stages of Moral Development According to Kohlberg (1981) 


eee 


Stage What is considered to be right 
eee. 
Stage il To avoid breaking rules backed by punishment, obedience 
Obedience and for its own sake, avoiding physical damage to persons and 
punishment property. 

orientation 

Stage 2: Following rules only when it is to someone’s immediate per- 
Instrumental sonal interest; acting to meet one’s own interests and letting 
purpose and others do the same; right is an equal exchange, a good deal. 
exchange 

Stage 3: Living up to what is expected by people close to you or what 


Interpersonal accord people generally expect of those in your role; being good is 
and conformity important. 


Stage 4: Fulfilling the actual duties to which you have agreed; laws 

Social accord and are always to be upheld except in extreme cases where they 

system maintenance conflict with other fixed social duties; right is also contribut- 
ing to society, the group, or institution. 


Stage 5: Being aware that people hold a variety of values and 

Social contract, opinions, that most values and rules are relative to your 
utility, individual group but should usually be upheld because they are the so- 
rights cial contract; some nonrelative values and rights like life 


and liberty, however, must be upheld in any society regard- 
less of the majority opinion. 


Stage 6: Following self-chosen ethical principles; particular laws or so- 
Universal ethical cial agreements are usually valid because they rest on such 
principles principles; when laws violate these principles, one acts in ac- 


cordance with the principle; principles are universal prin- 
ciples of justice; the equality of human rights and respect for 
the dignity of human beings as individual persons; the 
reason for doing right is the belief, as a rational person, in 
the validity of universal moral principles and a sense of per- 
sonal commitment to them. 


Note: Stages 5 and 6 are not distinguished for research purposes; there is also a 
transition stage (e.g., 2/3, 3/4) between each of the stages. 


variety of viewpoints adds to students’ cognitive conflict and thus is in- 
strumental to growth. It is assumed that the teacher’s answers will reflect a 
higher stage of moral reasoning and serve as a model for the most mature 
students in the group, thereby promoting growth. 

Most subsequent curricula inspired by Kohlberg’s theory have used a 
similar Socratic pedagogy. In his summary of research in values clarifica- 
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tion and moral development curricula, Lockwood (1978) analyzes 11 moral 
education studies, dividing them between those that employ direct moral 
discussion and those that employ psychological activities (e.g., peer counsel- 
ing). The direct discussion approach generally produces significant develop- 
ment in moral reasoning. According to Lockwood, the best-designed studies 
showed a mean advance of almost one half stage over a semester or an 
academic year, as opposed to one-third stage advances achieved by the 
psychological awareness approach. All subjects, of course, did not advance 
consistently; some students developed, while others remained at the same 
stage. Lockwood explains that the mean developmental increases are in the 
Stage 2 to Stage 3 range. 

In the most recent review of moral education programs and curricular 
effects on participants’ moral reasoning, Higgins (1980) finds an increase 
between one third and one half stage in experimental groups. Both Higgins 
and Lockwood confirm that in interventions where psychological activities 
predominate (as opposed to moral discussion), moral growth, as measured 
by the MJI, is modest. 

In sum, then, there have been studies that used hypothetical dilemmas 
exclusively as the curriculum for moral education and others that have at- 
tempted to weave into the discussions activities that relate more generally 
to psychological growth—for example, peer counseling, cross-age teaching. 
Few have attempted to integrate moral education into an academic dis- 
cipline, and the results of such studies have been contradictory and incon- 
clusive. Lieberman (1979) looked at the effects on moral growth of a junior 
high school 10-week unit, Facing History and Ourselves; The Holocaust and 
Human Behavior, and found that the experimental group did not increase 
in moral judgment significantly more than the control group, although sig- 
nificant increases in interpersonal awareness stage were found for the ex- 
perimental group. Similarly, in two American history classes taught by 
Ladenburg (1977) and his colleague Oakes at a Massachusetts high school— 
where role-playing of historical situations and engagement in hypothetical 
direct moral discussion were featured—neither group showed any moral 
growth. 

In contrast, a major study of secondary school moral discussion 
programs led by social studies teachers in the Boston and Pittsburgh areas 
as reported by Colby, Kohlberg, Fenton, Speicher-Dubin, and Lieberman 
(1977) found significant increases in moral judgment reasoning in com- 
parison with controls. This study provides a research base for hypothesizing 
that stage change can be realized through moral discussion within an 
academic discipline and through the collective role-playing that is a part of 
the study of fiction and drama in an English class that relies on the Socratic 
method. Additional support for hypothesizing stage change as a result of the 
investigator’s curriculum, with its blend of role-playing, discussion of moral 
and social issues in literary texts, theater games and drama, comes from the 
year-long course taught by Sullivan (1975) to juniors and seniors in high 
school as a social studies elective. His course included such elements as 
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moral psychology and philosophy, moral discussions, and counseling and 
empathy training, and it achieved a striking mean increase of one half 
stage. 

Teachers of literature are in an especially favorable position for assum- 
ing the responsibilities of moral education because so many poets and 
writers address themselves to moral issues. Sometimes justice is the focal 
point of the work, as in Lee’s To Kill a Mockingbird or in Lawrence and 
Lee’s Inherit the Wind; sometimes the moral issues are more peripheral, as 
in Huckleberry Finn or A Separate Peace. Each of these works is charged 
with moral problems which, through discussion, can promote the moral 
growth of young people. 

In creating this experimental curriculum in moral development and 
literature (Garrod & Bramble, 1977), Kohlberg’s (1981) cognitive-develop- 
mental approach to moral education was used because it is a conceptual tool 
which, in the face of conflicting values and moral systems, helps individuals 
to determine the fairest and most adequate course of action. Furthermore, 
Kohlberg’s theory can be used successfully by teachers to stimulate moral 
development without recourse to the “preaching” of moral content (which 
many educators regard as objectionable). 

The curriculum under study has the potential for initiating the process 
of examination and evaluation that leads to a careful reconsideration of in- 
herited values by the young. Whether those values are reaffirmed or re- 
jected, the student will have ceased to be a person whose morality is un- 
questioned. In this curriculum, literature is the vehicle for critical thinking 
and moral development. The following discussions of classroom process 
show how critical thinking and moral reasoning development were encour- 
aged in this curriculum. 


Classroom Process 

Blatt (1970) hypothesized that the cognitive-developmental approach to 
moral education might have practical as well as theoretical applications. He 
discovered that “moral discussion” was a powerful method of stimulating 
the development of moral reasoning. As a result, moral discussion has 
recently been introduced in many junior and senior high school social 
studies classrooms. However, the English curriculum used here takes a dif- 
ferent approach. Whereas the social studies curricula based on the cogni- 
tive-developmental mode have relied heavily on case studies and hypo- 
thetical dilemmas, in this investigator’s curriculum, literature is the source 
of discussion material. As in the social studies courses, moral discussion is a 
key strategy for stimulating moral development. The characters and situa- 
tions in this curriculum’s texts exhibit clearly delineated values and at- 
titudes. According to their established values, these fictional characters 
make decisions whose consequences they must endure for good or for ill. 

When the quality of the literature is high, fictional characters manifest 
points of view as valid as those held by members of the discussion group. A 
significant degree of role-taking (the ability to take on another’s social 
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perspective) is required if students are to appreciate these fictional perspec- 
tives. Students must also learn to listen carefully to one another and try to 
understand points of view they may regard as antagonistic to, or exclusive 
of, their own. Students were encouraged to show as much consideration for 
the characters in the selected works as they would extend to their peers in 
the discussion group. 

Normally, student role-taking in a moral discussion is confined to per- 
spectives held by members of the immediate group, but this curriculum in- 
vited students to assume the viewpoint of fictional characters as well. This 
role-taking was generated by posing questions designed to elicit value iden- 
tification as well as moral reasoning. Value identification is the role-taking 
process by which students attempt to identify the specific attitudes and 
values they perceive to be held by others. When a member of a discussion 
group is the object of this process, verification can always be obtained 
through feedback. When the process involves a fictional character, however, 
direct feedback is not possible, and discussion participants must identify the 
character’s values by a collective role-taking process in which students com- 
pare their impressions of the fictional character with those of their peers. 
Applying this process to the values and attitudes of fictional characters 
(which many experienced English teachers do as a matter of course) may 
impose stringent demands on the role-taking powers of discussion par- 
ticipants, but only in this way can characters be brought to life for students. 


Curriculum Implementation 
In practice, students in this study explored the moral problem as embodied 
in the text before moving on to a broader discussion. The text anchored the 
debate in the concrete and prevented it initially from becoming too hypo- 
thetical and abstract. 

Questions were phrased to permit responses at various stage levels. 
Thus, while the content may be similar in the responses of two different 
students (e.g., that one shouldn’t lie), the reasoning behind the response 
may indicate that the students are at different stages of moral development, 
agreeing on a particular choice but not on the reasons for making that 
choice. Consequently, students’ responses were carefully probed until their 
reasoning became clear to all members of the group. 

In some texts, the exact nature of the moral problem may be debatable, 
as in A Separate Peace, where the psychology of the characters is complex 
and the author’s interest is in psychology as well as justice. The develop- 
mental approach to moral education stimulates cognitive and moral dis- 
sonance as a means of promoting moral development rather than focusing 
on the “correct” identification of the moral problem. It is perhaps unneces- 
sary that every moral issue be sharply defined; it is enough that students 
find sufficient substance to trigger an interaction between their own moral 
viewpoint and that of the fictional character under consideration. Through 
interaction—between student and literature, student and student, or stu- 
dent and teacher—moral development can occur. 
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. Huckleberry Finn is an appropriate text to illustrate how a moral discus- 
sion approach can inform students’ reading and appreciation of a novel; it 
was selected to apply Kohlberg’s model to literature because Twain’s book is 
rich in themes of friendship. As the novel begins, Huck’s moral development 
has been temporarily checked by the smothering effects of Pap’s bullying, 
Tom’s domination, and the Widow’s motherly reforming. But Jim seems to 
intuit that caring for human beings often necessitates letting go of them— 
that showing respect for the dignity of the individual means honoring the 
right to make autonomous decisions. Of all the characters in the book, only 
Jim shows Huck this total acceptance. But freedom implies responsibility, 
and in the egalitarian environment of the raft and the river, Huck must 
bear the burden of, and the accountability for, his own moral judgments. No 
external authority can rescue him from his moral predicaments; he must 
decide for himself what is right, and it is this freedom to agonize over moral 
dilemmas that leads to Huck’s growth. 

The incidents occurring between chapters 16 and 31 are a constant 
source of cognitive and moral conflict for Huck, and the result seems to be a 
transition, in terms of Kohlberg’s model, from the preconventional to the 
conventional level of moral reasoning. In these chapters, Huck’s continuing 
dilemma is conflicting personal loyalties—to Miss Watson on the one hand 
and to Jim on the other. These loyalties suggest a Stage 3 orientation, and 
until the climactic episode at the end of chapter 33, Huck’s moral reasoning 
vacillates between Stage 2 and Stage 3. As his relationship with Jim con- 
tinues to blossom, he grows increasingly more concerned with Jim’s welfare 
at the expense of his own. This shift reflects a rejection of self-interest 
(Stage 2) as behavioral motivation in favor of trust, respect, loyalty, and 
gratitude—qualities chracteristic of Stage 3 moral reasoning. 

A caveat is in order. Obviously, a Moral Judgment Interview cannot be 
administered to Huck (or to any of the other characters in the selected 
texts), so unqualified stage claims cannot legitimately be made about the 
characters. It can nevertheless be argued that the story contains a range of 
reasoning typical of the transition from Stage 2 to Stage 3. Twain never 
makes the mistake, for example, of letting Huck argue the evils of slavery 
or the inalienable rights of humankind—concerns that arise at Stage 5. 
Without knowledge of stages and structures, he understood human nature 
well enough to avoid this sort of glaring incongruity and portrayed Huck’s 
level of moral reasoning consistently in the Stage 2-3 range, typical of a boy 
his age. The Adventures of Huckleberry Finn may have stylistic and narra- 
tive flaws, but in his young protagonist, Twain offers a true-to-life character 
for whom there are no shortcuts to moral development. 

To demonstrate more clearly the nature of the curriculum, a few sample 
chapter synopses and study questions from the unit on Huckleberry Finn 
are given in the appendix. The remainder of this article includes a detailed 
description of the method and results of the investigator's study using this 
curriculum. In addition, the strengths and weaknesses of this study are con- 
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sidered, as are suggestions for future research in moral development and 
the study of English literature. 


Method 
Subjects 


A total of 44 high school students participated in the study, 18 boys and 26 
girls. There were a total of 21 students in the experimental group and 23 in 
the comparison group. Only those who completed both pretest and posttest 
were included in the analysis. Both groups were grade 10 enriched English 
classes. Students opt for a grade 10 enriched English class on the advice of 
their junior high school English teacher; the main requirements for selec- 
tion are at least average achievement in English studies and the teacher’s 
assessment that the student is highly motivated. 

The groups appeared similar in several respects. Variation in SES, IQ 
scores, and achievement tests were similar for both groups. The sample 
reflected a wide variety of family backgrounds, ranging from parents who 
were university lecturers to those who were dockyard workers and to the 
unemployed. The average age of each group was estimated to be 15.5 years 
at the start of the academic year and 16.3 years at its termination. 


Design 


The design was a quasi-experimental one, closest to Campbell and Stanley’s 
(1963) tenth design, “the Non-equivalent Control Group Design.” As indi- 
cated above, students were not randomly assigned to groups; rather, the 
two teachers were randomly assigned to in situ classes—the two grade 10 
enriched classes. In other words, the assignment of the “treatment” to one 
group or another was random. The comparison group was taught by a dif- 
ferent teacher, and both she and the teacher-researcher used Socratic dis- 
cussion extensively in their pedagogy. The experimental and comparison 
groups were administered the Moral Judgment Interview as pretest (Form 
A) and posttest (Form B), in each case in written form: 


Time 1 Time 2 
Experimental Group 0 X 0 
Comparison Group “i od eee aty tp avo bel 0 or: 


The Treatment: The Curriculum 


The treatment was a year-long English literature curriculum constructed 
and taught by the investigator. The comparison group curriculum had some 
features in common with the treatment—for example, both classes studied 
a Shakespeare comedy and used a similar text of narrative and expository 
readings. The treatment group, however, studied A Separate Peace, Huck- 
leberry Finn, Antigone, David, and To Kill a Mockingbird, texts selected for 
themes appropriate to the concerns of adolescents and for the moral issues 
they embody. 
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Discussions occurred both in small groups and with the whole class; dis- 
cussion questions were addressed individually in homework assignments or 
taken up collectively after the reading of a text. Variety of approach to dis- 
cussions was a consistent consideration. Debates, formal and informal, cen- 
tered on issues and themes in the literature. Written work was assigned at 
different stages of the study and discussion of a text. Both spontaneous re- 
sponses and answers informed by group discussion were elicited through- 
out the year. Several feature and documentary films, integrated with the 
themes of the curriculum, were viewed and discussed. In addition, the cur- 
riculum for both groups included dramatization of scenes from Shake- 
speare’s Twelfth Night, theater games, and the staging of one-act plays in 
which all students performed. 


Measures 


The Standard Form Moral Judgment Interview and Scoring System (Colby, 
Gibbs, Kohlberg, Speicher-Dubin, & Candee, 1979) was used for both pre- 
and posttest assessments. For ali the students, Form A was used as a pre- 
test, Form B as a posttest. In light of the proven greater difficulty of Form 
B, a scale score was provided in the analysis. All students filled out the MJIs 
during class time. Oral probes were provided as students were administered 
the in-class written protocols. 

The Moral Judgment Interviews were scored by trained raters blind to 
both treatment group and times of testing. Scores reported are global stage 
scores indicating major stage usage and Moral Maturity Scores. Moral 
Maturity Scores (MMS) are weighted averages of stage use across the total 
interview. They range from 100 (Stage 1) to 500 (Stage 5), with a score of 
350, for example, indicating a protocol midway between Stages 3 and 4. The 
global stage score, true to developmental theory, presents the data in a 
more qualitative way; the MMS treats the data as a continuous variable. 


Results 

In keeping with other reports of intervention studies in cognitive develop- 
mental moral education, the analysis considers the data as a continuous 
variable and examines them in a structural way more true to its theoretical 
orientation. Thus the format for the presentation of results consists of an 
analysis of the data as a continuous variable followed by a structural anal- 
ysis of the data. In each analysis, the data are further examined by stage to 
assess the differential impact of the curriculum. 

The major finding of the study was that the curriculum proved effective 
in stimulating the growth of moral reasoning in preconventional students 
(those scoring below Stage 3) and that it led to increases in moral judgment 
maturity comparable to the largest increases obtained in other studies (e.g., 
Harris, 1976; Sullivan, 1975). However, when the increases in the experi- 
mental group are compared with those in the comparison group, there is no 
overall statistical difference. 

Because it was impossible to randomly assign students to treatment and 
comparison groups, a t-test was performed to test for initial equivalency of 
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the groups. The t-test indicated that the difference between the two groups 
on the pretest was statistically significant (¢=3.15; p=.003); the experimen- 
tal group exhibited more advanced moral reasoning (x=291.1) initially than 
did the comparison group (x=270.4). 

In an examination of the mean change in MMS from pretest to posttest 
for experimental and comparison groups, a t-test revealed that the 42.1 
MMS increase of the experimental group, when compared to the 32.4 MMS 
increase of the comparison group, was not statistically significant (t=2.10; 
n.s.). 

Further tests were carried out to see if the curriculum had a differential 
and significant impact on those who scored initially at different stages— 
that is, at Stage 2 and Stage 2/3, at Stage 3, and at Stage 3/4. Scores in- 
dicating stage mixture were not differentiated by major and minor stage. 
Thus scores of 2(3) and 3(2) were treated as equivalent (2/3), as were 3(4) 
and 4(3). 

In light of the large difference in variance, a separate variance estimate 
was used (Table 2), and the t-test revealed that the increase of 62.3 MMS of 
the Stage 2 and 2/3 students in the experimental group was significantly 
larger than the advance of 39.4 MMS of the comparison group (¢=3.10; 
p=.008). This finding is particularly striking in that it indicates that the 
curriculum was highly effective in stimulating the development of moral 
reasoning in students at Stages 2 and 2/3. 

A similar analysis of change scores (Table 3) was made for those from 
both groups who scored at Stage 3 on-the pretest. Despite the statistically 
insignificant difference of 13.3 MMS in mean increase between those in the 
experimental group who scored on the pretest at Stage 3 and those in the 
comparison group who scored at the same level, the direction of the dif- 
ference was markedly in favor of the experimental group. 

An analysis of the results of those scoring initially at Stage 3/4 is not 
presented in full because the number of students from both groups was 
small (five) and the increases modest (7.5 MMS gain with the experimental 
students and 4.4 MMS with the comparison students, respectively). In addi- 
tion, a two-way analysis of variance was carried out on the data using sex 
and group as factors, with no significant differences or interactions found. 


Table 2 
Change Scores for Stages 2 and 2/3 Students 


Number of Mean Standard t-value 
Cases Increase Deviation (2-tail prob.) 
Experimental 5 62.3 6.1 
3.10% 
(p = .008) 
Comparison 12 39.4 23.7 


*Based on a separate variance estimate. 
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Table 3 
Change Scores for Stage 3 Students 


SS ee eee ee ee 


Number of Mean Standard t-value 
Cases Increase Deviation (2-tail prob.) 
Experimental 13 42.4 27.3 
aad 
(n.s.) 
Comparison 9 29.1 27.5 


In Table 4 the strong upward trend in the experimental group is clearly 
evident. The left-hand side of Table 4 reveals dramatically that in the ex- 
perimental group all who started at Stage 2/3 (five students) went up a full 
stage. Of the 13 students who started the year at Stage 8, only two failed to 
move upward, and 11 (84.6%) moved up a half-stage. Those already at 
Stage 3/4, few in number (three), did more poorly—one regressed a full 
stage, one regressed a half-stage, and one remained stationary. No one 
reached Stage 4. These findings suggest the powerful effect of the cur- 
riculum on students at two stages: It had maximum impact on Stage 2/3 
students and a considerable effect on those pretesting at Stage 3. Of all the 
students in this group, 95.3% finished the treatment at Stages 3 and 3/4. 
The regressions may be due to scoring error or may be the result of ad- 
ministering the MJI in written form. In such circumstances, given the ab- 
sence of probing and possible student inertia, Higgins (1980) reports that 
regression is not unusual. In contrast, the comparison group results (right- 
hand side of Table 4) indicate that only 78.2% finished their course at 
Stages 3 and 3/4 and that 27.1% failed to advance beyond Stage 2/3. As in 
the experimental group, there was no advance for those who scored initially 
at Stage 3/4. 

Table 5 reports patterns of stage change from pretest to posttest for the 
experimental and comparison groups. It can be seen that 23.8% of the ex- 
perimental group gained a full stage, as opposed to only 8.7% of the com- 
parison group who did so. However, the movement up a half stage was 
remarkably similar—52.4% in the experimental group and 52.5% in the 
control group. 

Only two subjects in the experimental group showed a decline in moral 
stage; in the comparison group, three subjects showed such a decline. In the 
experimental group, 14.3% remained at the pretest stage as opposed to 
26.1% in the comparison group, but changes in the experimental group 
generally followed the pattern of sequential and step change found in nat- 
ural development. The analysis revealed no statistically significant differ- 
ences in patterns of stage change for the two groups. ane 

Completing a structural analysis of the data of those who initially scored 
at Stages 2 and 2/3, a significant difference appears between comparison 
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Table 5 
Structural Analysis of Stage Change: 
Experimental Versus Comparison 


Group -1 -1/2 0 +1/2 +1 Total 
Experimental 1 1 ‘6 11 5 21 
(4.8%) (4.8%) (14.3%) (52.4%) (23.8%) (47.7%) 
Comparison 0 3 6 12 2 23 
(0.0%) (13.0%) (26.1%) (52.2%) (8.7%) (52.3%) 
Total 1 4 fe) 23 7 44 
(2.3%) (9.1%) (20.5%) (52.3%) (15.9%) (100.0%) 


and experimental groups (X2(5)= 10.12; p=.0063). As exhibited by Table 2 in 
the analysis of the data as a continuous variable, Table 6 illustrates the 
statistically significant growth of the Stage 2 and 2/3 students. 

Similarly, a structural analysis of those who scored at Stage 3 initially 
indicated a much greater DENN in the experimental group than in the 
comparison group. While ie (2)=4.90 with a significance level of .08—that 
is, it is not quite significant—the number of half-stage advances in the 
treatment group (84.6%) was far greater than the number of half-stage ad- 
vances in the comparison group (44.4%). Thus the curriculum proved effec- 
tive for those who started at Stage 3, as it did for those who started at Stage 
2/3. 

The small number of those scoring on the pretest above Stage 3 renders 
a structural analysis of the data on these students uninformative. 


Discussion 
The present study examines the effects on moral reasoning of an English 
literature curriculum. The major finding was that the curriculum proved 
highly effective in stimulating the growth of moral reasoning in preconven- 
tional (Stage 1-2) subjects. The curriculum also had a considerable impact 
on those who initially scored at Stage 3. An important finding was that the 


Table 6 
Stage Changes for Stages 2 and 2/3 Students 


0 +1/2 +1 Total 
Experimental 0 0 5 5 
(0.0%) (0.0%) (100.0%) (29.4%) 
Comparison 2 8 2 12 
(16.7%) (66.7%) (16.7%) (70.6%) 
Total 2 8 7 17 
(11.8%) (47.1%) (41.2%) (100.0%) 


aN 
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increase in moral reasoning in the experimental group was comparable to 
the largest increases in other studies (Blatt & Kohlberg, 1975; Dozier, 1974; 
Harris, 1976; Sullivan, 1975). However, when the increase in the experi- 
mental group is compared with that in the comparison group, there is no 
overall statistically significant difference. 

More specifically, in qualitative terms and in stage breakdown, all ex- 
perimental students who initially began the program at Stage 2 and Stage 
2 advanced a complete stage. The advance of those in the experimental 
group who started at Stage 3 (85% moved up a half-stage) approached sig- 
nificance. Thus the curriculum appeared to have a marked effect on those 
who scored at the lower end of the stages represented in the treatment 
group; it proved ineffective with those already at Stage 3/4. Overall, the ex- 
perimental group advanced a mean of 42 MMS points in comparison to the 
gain of 32 MMS points in the comparison group. 

Consistent with developmental theory, almost all observable movement 
was toward the next stage, although 11% of the students regressed. These 
regressions may be due to administration of the MJI in written form—not 
the best method because students are more likely to respond cryptically 
with responses that would benefit from probing. Both raters, however, com- 
mented on the fullness of all the protocols in this study and remarked on 
their consequent confidence in scoring. 

The discussion focuses on the nature of the developmental gains and 
then assesses the extent to which the results fitted the investigator’s clinical 
sense of what was occurring. The full-stage gains in the experimental 
preconventional students (below Stage 3) is perhaps attributable to the con- 
tent of the curriculum, with its emphasis on issues of friendship and affilia- 
tion—issues most pertinent to adolescents at Stages 2/3 and 3—but not as 
conducive to moving a Stage 3 student toward a societal perspective, the 
hallmark of Stage 4. Perhaps, too, the presence of many conventional stu- 
dents in the class discussion, in keeping with the theory of arousing cogni- 
tive dissonance by exposure to higher-stage arguments, is instrumental to 
growth for these preconventional subjects. In addition, the preponderance 
of girls in the experimental group, with their relational orientation, might 
make marked gains in Stage 4 reasoning, as Kohlberg has measured it, un- 
likely (see, for example, Gilligan, 1982). While the discussion questioning 
moved from the text to exploration of general principles (e.g., Is hanging 
ever right?), it is also possible that moral and social problems enshrined in 
literature invite a more contextual moral consideration. They demand a 
careful reflection on motivation and consequences in a specific context and 
thus may not stimulate abstract philosophic discussion of principle as much 
as a psychological discussion of relationships and responsibilities. 

This investigator did not explore polarized moral options with his stu- 
dents because in literature of quality the moral concern is inevitably more 
complexly presented. Rather, he scrutinized the moral/social view in a par- 
ticular literary context, which was subsequently related to students’ ex- 
perience. It is debatable whether such an approach, undertaken by many 
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excellent English teachers, is likely to take students beyond conventional 
reasoning. This may help to account for the dearth of Stage 4 subjects which 
is surely attributable to the chronological age of this study’s subjects as well. 
In sum, chronological age, the absence of concentration on justice issues, 
and the focus on psychological understanding of motivations and conse- 
quences may be less likely to move students toward Stages 4 and 5 of moral 
reasoning than are the hypothetical moral dilemma discussions of many 
proven studies (Harris, 1976). 

These findings coincide with reviews of moral interventions by Lock- 
wood (1978) and Higgins (1980), who report that the greatest gains are in 
the Stage 2 and Stage 3 range. Only Harris (1976) was successful in achiev- 
ing significant increases in which posttest means were above 300 MMS. A 
consistent finding is the variability of treatment effect on students at dif- 
ferent stages. Neither the direct discussion approach nor the psychological 
approach to moral education has so far proved instrumental in stimulating 
reasoning beyond Stage 4 with adolescent populations. 

The findings in the experimental group did match the researcher’s clini- 
cal sense of what was occurring. The students were amazingly vociferous 
and argumentative. An abundant variety of moral reasoning stages was ap- 
parent in the class, and students were intensely involved in discussion. The 
atmosphere in the classroom was marked by increasing openness and a 
deepening engagement in literature, ideas, and self-exploration. Through- 
out the course of the year, and without apparent provocation, students com- 
mented that they had reassessed their position and reasons from those 
expressed on their pretest MJI; they seemed more ready to claim the new 
stance as their own. Honest exploration of viewpoint and feeling, in oral 
and written form, was characteristic of the experimental students. Discus- 
sion of issues of friendship particularly elicited lively and frank debate. 
Should friends tell each other their most secret feelings and leave them- 
selves vulnerable? Girls constantly advocated openness (“so that something 
can come of the relationship”); boys argued that being totally open was self- 
indulgent, led to dependency, and inhibited the development of autonomy. 
Such encounters were by no means unusual. The investigator felt confident 
that his students were growing in moral sensitivity and social under- 
standing; he was less certain about whether a “vertical” measurement such 
as the MJI, with its strong cognitive orientation, could capture what he per- 
ceived as the students’ steady growth. 

Methodological weaknesses of the study must be acknowledged. There 
are undeniable threats to internal and external validity in the design. Given 
the lack of randomization of students to groups, the absence of initial 
equivalency, the small sample size, the use of MJI in written form, and the 
fact that the investigator taught only the experimental group, the results 
must be evaluated cautiously. 

With respect to gains in the comparison group, unquestionably one of the 
strengths of the design of this study, as Higgins (1980) endorses in her 
review of moral interventions, is the duration of treatment—a full academic 


75 


A. Garrod 


year. Yet what constitutes a strength may also pose a threat to internal 
validity, for over such a time period it is possible for other factors to intrude. 
For example, DiStefano (1978), in her study of “The Psychology of Ethics in 
Relationships,” achieved gains of 42 MMS in the experimental group and 26 
MMS in the comparison group—similar to the results of this study. Because 
all her study’s subjects were members of a democratic alternative program 
involving moral discussion and democratic decision making, it was hard to 
disentangle the effects of the intervention from the effects of the alternative 
program. 

In this case, the teacher for the comparison group was a popular, lively, 
Socratic performer. The expectation was that the two classes would be 
equivalent, yet statistical analyses found them not to be so. It is possible, 
then, that there was a selection-maturation interaction; perhaps in light of 
their pretest inequivalency, the natural rate of change in the two groups 
would have been different even in the absence of the experimental treat- 
ment. 


Directions for Future Research 

Any further study undertaken along these lines should be conversant with 
the various threats, particularly to internal validity, that have been pre- 
sented. The multifaceted nature of the treatment, with the design weak- 
nesses already explained, render it hard to argue with certainty for the 
effects of the curriculum and impossible to make differential claims about 
the relative impact of parts within the curriculum. Yet, with the study by 
Colby et al. (1977), it offers tentative evidence that moral reasoning stage 
advance can be accomplished within an academic English literature course. 

However, to be concerned only with the raising of MMS is too limited a 
goal. If moral development and moral education are more broadly defined, 
as Kohlberg (1979), Gilligan (1977, 1982), and Rest (1980) suggest, it must 
also cover moral sensitivity, moral character, and moral values and be- 
havior. An English literature study, in that case, would wish to look at 
developmental variables other than just the Moral Maturity Score. Ego 
development may be a suitable variable. Loevinger (1976) has constructed a 
theory of ego development in which patterns of cognitive interpersonal and 
ethical development form successive and hierarchical world views: “In 
general, ego development is marked by a more differentiated perception of 
one’s self, of the social world, and of the relations of one’s feelings and 
thoughts to those of others” (Candee, 1974, p. 621). Fostering an under- 
standing of concepts of self and the social world falls reasonably within the 
mandate of an English teacher concerned about moral growth. 

In addition to developing other indices of change, a more rigorous study 
examining moral and ego development might look at the impact of different 
parts of a curriculum as manifested in writing samples throughout the year 
because connections between moral and ego stage and students’ writing— 
critical, imaginative, reflective, and autobiographic—could help us to better 
understand the process of change and aid curriculum development. 
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Appendix 
A Sample of Curricular Material from the Huckleberry Finn Unit 


Chapter 15 finds the companions separated in the fog and then reunited. 
The trick that Huck plays on Jim backfires and reveals a new dimension, a 
new depth, to the friendship. Huck says, “It was fifteen minutes before I 
could work myself up to go and humble myself to a nigger; but I done it, and 
I warn’t ever sorry for it afterward, neither.” 


Suggestions for Exercises 


A. Invite students to explore Huck’s reasoning during the 15 minutes he 
spent making up his mind to apologize, then have them write a script, 
with imaginary dialogue, detailing the various arguments that cul- 
minate in the apology. 


Suggestions for Study Questions 


A. Why is Huck so surprised by Jim’s indignant response to his innocent 
joke? 

B. Why does it take Huck a full 15 minutes of internal debate before he can 
bring himself to apologize? 

C. What considerations hold him back? 

D. As a friendship deepens, does the obligation to apologize for unkind or 
thoughtless acts or words become greater or lesser, or does it remain 
constant? 

fk. Discuss the development of Huck’s role-taking ability by comparing the 
time when he does not apologize (the snakeskin incident in chapter 10) 
with the time that he does (after the separation in the fog). 


Chapter 16 deals with the first of two major moral crises that confront 
Huck (the second arises in chapter 31). Huck’s dilemma involves the con- 
flict between his friendship for Jim and his perceived obligations to society 
(and to Miss Watson in particular). His reasoning in resolving this dilemma 
is presented in the middle of the chapter and is a classic example of Stage 2 
instrumentalism: doing “whichever comes handiest at the time.” 


Suggestions for Study Questions 


A. What factors influence Huck’s initial decision to turn Jim over to the 
authorities? 

B. What considerations encourage him to change his mind when he decides 
to lie to the bounty hunters? 

C. Isn’t it wrong to lie to them? Is it wrong to lie under all circumstances? 
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D. What are the moral implications of the rule Huck embraces when he 
decides in the future he will do “whichever comes handiest at the time”? 
How does this rule solve the problem of right and wrong for Huck? 


Chapter 31 contains the moral climax of the novel. The comparison of 
Huck’s reasoning in chapters 16 and 31 reveals Huck’s moral development: 
Faced with essentially the same dilemmas, his moral reasoning moves from 
Stage 2 to Stage 3 as a result of the constant cognitive conflict besetting 
him. As expected, Huck’s moral reasoning vacillated between Stage 2 and 
Stage 3, but the dilemma posed by Jim’s flight has at last been resolved by a 
more adequate moral rule. The Stage 2 instrumentalism which charac- 
terized Huck’s reasoning in chapter 16 has now given way to a broader and 
less self-centered perspective: Huck’s new principle might be termed 
“friendship first.” The morally right is now defined as that which is com- 
patible with friendship, and this broader social perspective has been made 
possible by his developing ability to take the role of others—to differentiate 
the self from the needs of self. Once he can make this differentiation, Huck 
is capable of distinguishing between others’ needs and his own. 


Suggestion for Exercises 


A. Write the dialogue for a scene in which Huck and an imaginary charac- 
ter argue the morality of his decision to tear up the letter and set Jim 
free. 


Suggestions for Study Questions 


A. Should Huck have torn up the letter? 

B. Can his reasons for tearing it up be derived from the pragmatic rule for- 
mulated in chapter 16: doing “whichever comes handiest at the time”? 

C. Does the principle underlying Huck’s decision to tear up the letter pro- 
vide a more adequate basis for decision making than the principle of 
doing “whichever comes handiest at the time”? 

D. Compare Huck’s commitment (loyalty) to Jim with their earlier relation- 
ship. What differences do you see, and how do you account for them? 
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Problem Solving Among 
Pairs of Achieving and 
Learning Disabled Students 


The present investigation of children’s problem solving strategies and task perceptions 
sought to apply a processing model to examine individual differences in children’s prob- 
lem solving. Fifty-six pairs of 10-year-olds classified gifted, learning disabled or average 
achievers engaged in a river-crossing problem solving task using think-aloud procedures. 
Learning disabled (LD) students were further classified as either helpless or mastery 
oriented. Transcriptions of videotaped sessions were analyzed for use of heuristics and af- 
fective variables as well as for sequences of moves toward solution. While learning dis- 
abled pairs differed from gifted and average pairs in solution, frequency of recognition 
errors, bizarre moves, and aspects of strategy use, no differences were found between 
Gifted and Average groups. For the LD groups, deficits were identified in cognitive proces- 
ses of comparison and evaluation, while metacognitive strategies for rereading or draw- 
ings indicated no differences compared with achieving students. No differences were 
found based on helplessness classification or for task-related affect and interaction. Im- 
plications for instruction pointed to the usefulness of pair procedures to assess problem 
solving among disabled learners. 


In special education, interest in the problem solving abilities of learning dis- 
abled children has coincided with developments in cognitive psychology. 
Theories of information processing have contributed new perspectives on 
the role of metacognitive strategy deficits in such areas as problem iden- 
tification and monitoring to explain the achievement of learning disabled 
children (Torgeson, 1986). At the same time, methodological advances re- 
lated to the study of cognitive processes have been successfully applied to 
the study of learning disabilities. Protocol analysis and the use of problem 
solving pairs have begun to be successfully incorporated into research, as- 
sessment, and teaching practices for the learning disabled (Gelzheiser & 
Shepherd, 1986). 

Much of the comparative problem solving research with achieving and 
learning disabled children has employed academic tasks (Stone & Michals, 
1986). While this approach has provided insight into aspects of problem 
solving deficits for the learning disabled, it has also posed difficulties. It has 
not been possible to adequately match comparison groups on knowledge 
prerequisites when investigating academic problem solving. An alternative 
approach has employed a cognitive processing model based on tasks which 
require little academic background knowledge. It was the aim of the present 
study to investigate learning disabled children’s use of problem solving 
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strategies based on a model of cognitive processing. Before detailing this al- 
ternative, relevant research on the problem solving skills of learning dis- 
abled children is reviewed. 

It would appear that disabled learners tend to disregard relevant infor- 
mation, do not adequately understand problems, do not use systematic plan- 
ning, persist with unsuccessful alternatives, and do not evaluate their 
conclusions (Stone & Michals, 1986). Nevertheless, while the problem solv- 
ing skills of learning disabled students have been studied over a variety of 
tasks from recall of written information and game rules to puzzle assembly, 
performance results have been somewhat mixed (Havertape & Kass, 1978). 

Kavale (1980) sought to evaluate problem solving strategies on a read- 
ing comprehension task where students were to give reasons (aloud) for ac- 
cepting or rejecting choices listed in a multiple choice format. While poor 
readers were three reading levels below that of average readers, there was 
no identification of the reading difficulty level of the material compared 
with average readers. Poor readers produced more comprehension errors, 
used reasoning strategies less frequently, and produced a higher rate of 
word recognition errors. It is difficult to ascertain the nature of the reason- 
ing strategy deficit in a task environment where the deficit skill is integral 
to task completion. 

From another perspective, Kotsonsis and Patterson (1980) sought to 
evaluate cognitive monitoring in learning to play a game. Each child was 
given one game rule at a time and was to signal when he or she thought he 
or she knew how to play. Learning disabled children, despite ability to recall 
game rules and attention to the task, judged themselves ready to play with 
significantly fewer rules available compared with average achievers. Al- 
though it was concluded that learning disabled children appeared less sensi- 
tive to the adequacy of information received and what was needed, there 
was no student verbalization to evaluate aspects of self-monitoring process- 
es or student task perception. 

Using academic and nonacademic tasks, Dweck (1975) and others (But- 
kowsky & Willows, 1980; Diener & Dweck, 1978) arbitrarily manipulated 
success and failure feedback to underachieving and achieving children. On 
receiving corrective feedback during problem solving, underachieving help- 
less children were characterized by greater use of disconfirmed hypotheses 
and ineffectual hypotheses compared with average achieving peers. The 
helpless children tended to discount their successes, were pessimistic about 
outcome, and made attributions for their difficulties to uncontrollable fac- 
tors. Although dramatic differences have emerged in problem solving styles 
for the two groups, the authors have noted that these studies have included 
a strong evaluative component established by experimenter feedback of 
“right-wrong.” Thus it remains unclear what and how problem solving pro- 
cesses were used effectively by able and disabled learners and to what ex- 
tent on-going perceptions about one’s performance or ability affect problem 
solving. Nevertheless, motivational factors and the syndrome of learned 
helplessness have been implicated as a significant influence in problem solv- 
ing effectiveness of learning disabled students (Licht, 1984). 
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Harris (1982) addressed the issue of verbal self-monitoring and cognitive 
training as it affects the problem solving of young children. Learning dis- 
abled and normal achieving eight-year-olds attempted a puzzle task where 
one puzzle piece was rigged to fit incorrectly. LD students scored sig- 
nificantly lower in time on task than normally achieving children, while no 
differences were noted in the number of problem solving strategies used by 
both groups. At the same time, the proportion of task irrelevant private 
speech was found to be significantly higher for learning disabled than for 
achieving students. In addition, learning disabled students were more likely 
to use negative affective statements as part of the task monologue. Harris 
hypothesized that task persistence was related to task-irrelevant private 
speech, and subsequent cognitive training both increased use of task- 
relevant speech and persistence time. Nevertheless, the relationship be- 
tween use of problem solving strategy and monitoring needs further clar- 
ification. In the Harris (1982) study, both groups of young children dis- 
played similar use of a variety of observed strategies, and most of the chil- 
dren completed all of the puzzle that was possible. 

It appears that cognitive monitoring has been a key area where problem 
solving differences have occurred. A consistent finding has been the use by 
learning disabled students of task irrelevant self-statements and negative 
self-evaluation in problem solving monitoring (Licht, 1984). Such negative 
evaluation tendencies might be expected for students who have histories of 
learning difficulties and who are characterized as helpless. When the prob- 
lem solving task environment has an academic focus managed by an adult 
evaluator, the influence of negative evaluation may be heightened. How- 
ever, in seeking to investigate the learning disabled child’s capability to use 
cognitive processes for problem identification planning, and evaluation, it is 
necessary to provide a task setting which minimizes associations with 
school failure and adult intervention in the monitoring process. In this 
respect, the model developed by Simon (1975, 1983) and others (Jeffries, 
Polson, Razran, & Atwood, 1977) has particular merit in studying the prob- 
lem solving of learning disabled students. 

Problem solving is characterized by cognitive processes which operate in 
sequence of search activities (Simon, 1983). A solver searches a mental 
problem space sequentially, gradually adding to the store of information 
about the problem and its solution by considering essential features and 
using heuristics. Heuristics are tentative, general decision processes which 
organize and narrow the search for a problem solution. Means-end analysis 
is a prominent heuristic by which the solver compares the current state 
with the goal state, identifies differences, and readjusts or sets subgoals in 
order to reduce differences. Within means-end analysis the solver also sets 
up a priority system to decide which subgoals should be implemented. Mem- 
ory processes are involved in storing information on those subgoals which 
have been met and those yet to be tried. 

Simon (1975), in characterizing problem solving as a process of search, 
focused on the use of the well-defined problem. Well-defined problems con- 
tain in the problem statement all the information needed to determine what 
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the solution should be and to evaluate progress toward solution. In addition, 
well-defined problems usually require a standard sequence of problem 
decisions or moves. Processing models have been applied to a variety of 
well-defined problems including the Tower of Hanoi, water jug problems, 
and river-crossing problems, in which all information concerning restric- 
tions and the goal state are specified (Jeffries et al., 1977; Klahr & Robin- 
son, 1982). Because the use of a well-defined problem incorporates the cog- 
nitive processes of interest in studying problem solving and deemphasizes 
academic background, it would appear especially suited for an investigation 
of learning disabled children (Spitz, 1987). 

The question of how effectively to record cognitive activity has been 
reviewed by Ericsson and Simon (1984). They have presented an overview 
of the validity of using verbal reports as data within the framework of infor- 
mation processing theory, and the use of concurrent, think-aloud reports 
has been widely accepted. Nevertheless, alternatives to think-aloud proce- 
dures have been suggested which maintain direct access to ongoing cogni- 
tive activity while allowing a more natural task environment for spon- 
taneous verbalization. Problem solving in pairs and in small groups has 
been considered in light of their advantages for verbalization (Noddings, 
1982; Whimbey & Lochhead, 1979). In particular, structuring the task for 
pairs seems well suited to eliciting decision making processes. Schoenfeld 
(1982) found that single-person situations were more apt to generate a set 
to produce something for the researcher, while two-person sessions eased 
pressure on subjects. When experimenter intervention was minimal, discus- 
sions between members of the pair regarding what to do next made the use 
of heuristics more explicit and observable so that the researcher was re- 
quired to make fewer inferences about how decisions were made. 

In summary this study sought to apply a cognitive processing model and 
solver pairs procedures to examine individual differences in children’s prob- 
lem solving strategies such as means-end analysis, problem understanding, 
use of task restriction rules, and use of subgoals. Because the study was con- 
cerned with motivational as well as cognitive aspects of problem solving, 
personal performance evaluations were examined in relation to effective use 
of problem solving variables, and an attempt was made to identify helpless 
student problem solving characteristics with a learning disabled sample. 


Method 
Overview 


Three groups of average achieving, learning disabled, and gifted students 
were videotaped during a problem solving task conducted in pairs. Learning 
disabled students were designated either helpless or mastery-oriented based 
on questionnaire results and teacher ratings. At task completion, children 
were interviewed concerning their reasons for solution and perceptions of 
performance. Aspects of problem solving were coded and analyzed from 
written transcriptions of videotapes. 
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Subjects 


112 students (90 boys, 22 girls), aged 10-0 to 11-6 years were selected from 
elementary schools serving predominantly suburban, middle-class neighbor- 
hoods in a large public school district in the Niagara region of southern On- 
tario. 


Gifted 

Initially 46 students were identified as gifted according to school admission 
criteria for enrichment programs. These students attended three enrich- 
ment classes which provided a minimum of half-time instruction in a 
segregated class setting. Students were categorized as gifted based on two 
of three subtest scores equal to or above 130 on the Canadian Cognitive 
Abilities Test. Teacher recommendation, student and parent inventories 
were also part of the school gifted identification process. Eleven male pairs 
and 12 female pairs were first identified based on student choice and 
teacher judgment of positive sociability for the pairs. 

Because sex composition of the learning disabled population favors 
males, a ratio of approximately 4:1 was maintained for the gifted group by 
finally selecting all of the 11 male pairs and randomly selecting three of the 
female pairs from the initial pool of 46 students. 


Learning Disabled 


Fifty-six students were selected based on school board nomination for 
specific learning disability classes. Requirements were based on IQ scores 
equal to or above 85 and on academic achievement at least one year below 
grade level in one or more academic areas. All learning disabled students 
between 10-0 and 11-6 years of age (mean age, 10-4) were selected from 
seven full-time, segregated classes for children with specific learning dis- 
abilities. Twenty-two male and six female pairs were formed based on stu- 
dent choice, teacher estimate of sociability, and similar helplessness scores. 
A helpless LD group consisted of 138 male and three female pairs; a mastery 
oriented LD group consisted of nine male pairs and three female pairs. 


Average Achievers 


Twenty-eight students with a mean age of 10-6 and helplessness scores of 8 
or above (Dweck, 1975) were selected at random from three grade 5 classes 
serving the same area from which gifted and learning disabled students 
were selected. Only those whose school records indicated achievement at 
grade level and no history of learning problems were selected for participa- 
tion. The average achieving group consisted of 12 male and two female 
pairs based on student choice. 


Measure of Learned Helplessness 


Research on helplessness in learning disabled children has identified the sig- 
nificant role of attributions in perceptions of failure (Diener & Dweck, 
1978; Dweck, 1975). Following assessment procedures developed in this re- 
search, the Intellectual Achievement Responsibility (IAR) Scale was ad- 
ministered to LD students two weeks prior to the experimental task 
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(Crandall, Katkovsky, & Crandall, 1965). Thirty-four items describe nega- 
tive or success achievement experiences that occur frequently in children’s 
daily lives. These are followed by attribution choices to either internal re- 
sponsibility (ability, effort) or external, environmental causes. Ten items of 
the LAR refer to failure experiences in which effort is an alternative. 

Based on the format developed by Dweck (1975) to identify helpless 
children, LD students were classified helpless who scored 7 and below on 10 
effort items of the IAR, while those scoring 8 and above were designated 
mastery oriented. In the present sample, the mean effort score for LD help- 
less (LDH) pairs was 4.04 (SD=2.12) with scores ranging from 0 to 7. For 
mastery oriented (LDM) pairs the average effort score was 8.12 (SD=.33), 
with scores ranging from 8 to 9. 

In addition a teacher rating scale adapted from Dweck (1975) was used 
to rate LD student helplessness. Fifteen items describe student behavior 
under failure and success situations on a five-point scale. Mean teacher 
ratings were 45.2 and 41.28 for LDH and LDM groups respectively, indicat- 
ing similar helplessness ratings by teachers for both LD groups. 


Task and Materials 


The problem was a cover story for children of the traditional river-crossing 
problem where four travellers must cross from one side of a river to the 
other, under certain constraints (Fletcher, 1968). The problem was pre- 
sented to all pairs by means of audiotape and in writing as follows: 
Farmer Jones went to the County Fair with his prize goat, his pet wolf and his wife’s 
prize pie. He had to cross a river in a small boat which fit only the farmer and one 
other object. If left together, the wolf would eat the goat, but not the pie. The goat 


would eat the pie if left alone with it. How should Farmer Jones work it out so that he 
can get all his possessions across the river? 


Procedures 


Three female experimenters, equally represented across the four student 
groups, conducted the sessions over a two-week period. Within each school, 
each pair was brought to a room which was set up with a table, chairs, cas- 
sette tape recorder for the children, and video equipment. It was explained 
that new school materials were being developed and it was necessary first to 
try them out with students to get their opinions. 

Before attempting the experimental task, children practiced thinking 
aloud together with a training task involving two number completions. 
Children heard the problem on audiotape and followed along on a problem 
card. The experimenter explained that once started the children would be 
on their own. Students were encouraged to work together and talk aloud 
about the problems. They were reminded to use the audiotape and problem 
card as often as needed. The experimenter played the practice tape and 
modeled a practice problem using audio tape checking, self questioning, goal 
stating, and drawing on paper. 
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Experimental Task 


Pilot testing revealed that 35 minutes was sufficient to allow students to 
generate alternatives or arrive at a solution. After 20 minutes, if no solution 
was achieved, one clue was given, “Remember, the farmer may take a pos- 
session with him when he goes back across the river.” This clue represented 
the point at which students often have greatest difficulty, a move which is 
necessary but which, in the short term, is a move backward, away from 
solution (Simon, 1975). There was no interaction between experimenter 
and pair until the pair signaled they had a solution. If an incorrect solution 
was given, the experimenter offered only the feedback “no, that’s not it.” 


Coding Procedures 


Verbatim written transcriptions of videotaped sessions included all dialogue 
and self-talk within pairs. As well, nonverbal behavior such as listening to 
the tape recorder or using worksheets was noted on the transcriptions. 


Coding Scheme 


An adaptation of the Jeffries et al. (1977) model of the problem solving 
process formed the basis of the scheme used to analyze pair protocols. Each 
protocol was parsed into three types of unit. Move units constituted dia- 
logue related to the selection of a move to one side of the river including 
evaluative discussion related to that move. State 0 units were those where 
students considered possible moves but were unable to select a move. State 
8 units were those which summarized or reviewed after a solution was 
achieved. Total units constituted the sum of the three unit types. However, 
only State 0 and move units were considered for analysis in this study. 

Move units were analyzed according to state and legality. According to 
Jeffries et al. (1977), a series of eight states are possible leading to the goal 
state of crossing the river. Each state represents one move. Moves at each 
state were legal if they conformed to the three constraints that neither goat 
and pie nor wolf and goat were to be left alone, and Farmer Jones was to 
cross with only one object. J/legal moves were those which violated any of 
the constraints. Because it was possible to continue to attempt a solution 
after making an illegal move, those subsequent moves were labeled recogni- 
tion errors. The recognition error path led to premature solution at State 4. 
State 3 was a crucial move because it required something new, returning to 
the left bank with an object. The remaining states required recursion while 
adhering to the three constraints. Unit coding involved identifying the state 
of the unit and whether the move was legal, illegal, recognition, or bizarre 
error. The latter moves involved alternatives completely outside the con- 
straints of the problem such as eating the pie or swimming across the river. 


State 0 Coding 


A coding scheme for problem solution processes at State 0 was developed for 
monitoring and heuristic activities as well as social aspects of problem solv- 
ing in pairs.1 Following Schoenfeld’s (1982) study of these units as transi- 
tion points in problem solving where difficulty may be encountered, it was 
of interest to identify types of strategy used as well as related affective and 


86 


Problem Solving in Pairs 


social variables. Three problem solving strategies and two social compo- 
nents were identified which were rated according to level of response as fol- 
lows: 

Considering moves—ratings (1 to 4) based on the degree to which prob- 
lem constraints were considered. 

Problem identification—ratings (1 to 4) based on the degree of recogni- 
tion of difficulty and specific action in seeking clarification of the problem. 

Representing the problem—ratings (1 to 3) based on the degree to which 
the pair seeks to draw or to take notes about the problem. 

Interaction style—ratings (1 to 5) based on the degree to which pair dis- 
cussion is mutual/independent, helpful/unhelpful in generating ideas, listen- 
ing, encouraging the other. 

Affect—ratings (1 to 3) based on the degree to which the pair expresses 
enjoyment, interest, a sense of accomplishment. 


Coder Reliability 


Two coders who were unaware of experimental group classification received 
instruction in the coding scheme, in two phases, over a seven-day period. In 
phase one, using training transcriptions of pairs not included in the study, 
coders achieved 90% agreement in identification of types of units and 
moves. Coders proceeded to score transcriptions for only these phase one 
variables. Fourteen randomly selected protocols, representing 25% of the 
sample, were coded by both coders. Percentage of agreement between 
judges for the phase one variables ranged from .93 to .97 across the unit and 
move variables. 

In phase two training, coders achieved a minimum of 90% agreement for 
ratings of protocols and for scoring of posttask performance evaluations. 
After training, phase two scoring was undertaken. For the previously ran- 
domly selected protocols, correlations between scorers ranged from .81 to 
.92 for the phase two variables. 


Results 

At the outset, groups differed significantly in solution of the problem 
(X2=30.77, df=21, p<.001). While all Gifted pairs and most (11 of 14) 
Average pairs solved the problem, few of the LD pairs (4 LDM, 1 LDH) did 
so. There were few or no claims that solution was achieved through State 3 
for Gifted and Average pairs. In contrast, LD groups made claims for solu- 
tion at State 1 and State 2, with significance at State 2 (X2=18.32, df=9, 
p<.05). At State 4, while all groups made some claims for solution, the LD 
groups made significantly more claims (X2=34.16, df=21, p<.05). It will be 
recalled that State 4 represents two solution paths based on recognition er- 
rors. A closer analysis of move patterns will clarify the underlying basis for 
group differences in claims. 


Move Types Across Problem States 


Before analyzing group differences for each type of move across problem 
states, an initial MANOVA was performed to control overall alpha level 
within the study (Pedhazur, 1982). The Wilks’ lambda approximate F value 
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was statistically significant for groups on all the move-related variables an- 
alyzed simultaneously, F (12,130)=3.74, p<.001. 

Table 1 shows the group means standard deviations and F values for the 
four categories of move and indicates mean codable units produced in pair 
dialogues and mean frequency of problem solving impasse (State 0). 

Significant group differences were noted for total units of dialogue pro- 
duced. While LD groups did not differ from each other, they produced more 
total units than Gifted or Average pairs who did not differ. It would appear 
that while the LD pairs did not achieve solution, they engaged in more prob- 
lem discussion than the two non-LD groups. However, no group differences 
were observed in instances of State 0 discussion (points of problem solving 
impasse) or frequency of legal moves. 

Group differences were significant for all types of nonlegal moves (il- 
legal, recognition, and bizarre errors). Table 2 summarizes mean moves by 
student group across six problem solving states. 

Following the Jeffries et al. (1977) model, States 2, 6, 7 for illegal moves, 
States 1, 6, and 7 for recognition errors, and State 7 for legal and bizarre 
moves were not included in the analysis of moves across problem states. 
Either the type of move was not possible at that state, or there were too few 
moves made by groups. A groups by repeated measures (State) analysis was 
performed on frequency of each of the four move types in order to examine 
differences in group profiles as a function of problem state. Main effects due 
to problem state were significant for all types of move: F legal (5,48)=29.94, 
p<.0001; F illegal (3,50)=9.77, p<.0001; F recognition (5,480)=8.79, 
p<.0001; F bizarre (4,49)=7.36, p<.0001. Interaction effects for problem 


Table 1 
Group Means, Standard Deviations, and F Values 
for Total Move-Related Variables 


Variables Gifted LDH LDM Average F Value 

Units 13.07 25.92 23.44 19.07 5.52% 
(6.37) (8.71) (8.17) (10.91) 

State 0 3.64 5.25 5.63 5.29 2 
(3.59) (3.28) (3.20) (5.37) 

Legal Moves 7.50 6.08 7.00 9.64 2.48 
(1.74) (3.70) (4.10) (3.91) 

Illegal Moves 21 3.17 175 1.29 Late oe be 

(.58) (2.59) (1.77) (1.73) 

Recognition 64 4.58 4.06 1.79 535% 

Errors (1.65) (3.12) (3.57) (3.17) 

Bizarre Moves 1.14 7.33 5.00 1.14 6.020 
(1.79) (6.20) (4.87) (1.35) 

*p <.005 

**9 < .001 
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Table 2 
Group Means and Standard Deviations 
for Type of Move Across Problem States 


Move Type State 1 State 2 State 3 State 4 State 5 State 6 
Gifted 
Legal 2.29 1.43 at 1.00 64 93 
(.91) (.85) (.42) (.39) (.49) (.27) 
Illegal .14 ~ .0O .07 .00 - 
(.53) 3 (.00) (.27) (.00) = 
Recognition Error - 44 14 36 .00 _ 
. (53) (.36) (.84) (.08) = 
Bizarre 57 36 07 14 .00 .00 
(.76) (.63) (.27) (.36) (.00) (.00) 
LD Helpless 
Legal 3.42 1.67 42 25 Hy sd 
(1.83) (1.56) (.79) (.45) (.39) (.39) 
Illegal 1.92 - .25 .67 17 ~ 
(1.98) = (.62) (.78) (.39) - 
Recognition Error -- 1.75 .20 1.95 45 - 
- (1.66) (.40) (1.31) (.90) - 
Bizarre 2.75 2.42 16 1.42 oz 25 
(2.96) (1.73) (.58) (1.68) (.39) (.45) 
LD Mastery 
Legal 3.94 2.69 aa ke) 13 .00 13 
(1.98) (1.77) (.34) (.34) (.00) (.34) 
Illegal 1.31 -- 13 31 .00 - 
(1.50) - (.34) (.60) (.00) - 
Recognition Error - 94 94 2.00 13 - 
- (1.18) (1.48) (1.93) (.34) - 
Bizarre 1.31 1.38 31 .88 38 .50 
(1.81) (1.89) (1.01) (.96) (.81) (.89) 
Average 
Legal 4.29 2.43 79 .79 $e 64 
(2.20) (1.45) (.43) (.43) (.46) (.50) 
Illegal 7 - 36 .36 .00 - 
(1.28) - (.75) (.63) (.00) - 
Recognition Error - .50 .50 64 14 36 
-- (1.09) (1.16) (1.15) (.36) - 
Bizarre 79 21 07 07 00 00 
(1.12) (.42) (.27) (.27) (.00) (.00) 
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state by group were significant for F legal (15,133)=2.60, p<.002 and F rec- 
ognition (15,133)=2.14, p<.01. 

Based on simple effects analyses (Table 3), there was a tendency for 
Average and LD pairs to select more legal moves at States 1 and 2 than did 
Gifted pairs. From States 3 to 7 the trend was for Average and Gifted 
groups to produce more legal moves than LD groups who did not differ from 
each other. 

For illegal moves, State 1 was the only state in which significant dif- 
ferences were noted, with both LD groups producing more moves than 
Gifted or Average pairs. States 1, 2, and 4 produced significantly more bi- 
zarre responses for LDH pairs than for either Average or Gifted pairs. For 
recognition errors, LDH pairs produced significantly more such moves than 
Average or Gifted groups at State 2. At State 4, both LD pairs offered more 
recognition errors than Gifted pairs, with LDM pairs also producing more 
than Average pairs. In Figure 1 the group profile for recognition errors is 
representative of the pattern for nonlegal moves (illegal and bizarre) across 
problem states. In the early States 1 and 2, while both LD groups produced 
almost twice as many nonlegal moves as Average pairs, the Average pair 
responses were also twice as frequent as those of Gifted pairs. At State 4—a 
pivotal state which requires a shift in strategy—the nonlegal responses of 
both LD groups were more than twice those of non-LD groups. For legal 
moves the pattern was reversed. While all pair groups produced more legal 
than nonlegal moves at States 1 and 2, Gifted pairs made the least legal 
moves. On the other hand, for States 3 through 6, Average and Gifted pairs 
maintained legal move production while LD production of legal moves was 
negligible. It appeared that while Gifted and Average pairs became more 
selective in their choice of legal moves, LD pairs shifted to nonlegal alterna- 
tives or returned to an earlier problem state. This conclusion is supported 
by the significantly greater LD claims for solution at early problem states. 


Table 3 
F Value Results for Group Simple Effects Analysis of Move Data 


State Legal Moves Illegal Moves Recognition Errors Bizarre Errors 


1 3.62* 4.17* - CW (he 
2 3.83* - 4.54** 7.47*** 
3 12.62*** 1.31 1.96 .49 
4 16.04*** 2.19 5.24** be Bate piste 
5 Slee oe 2.72 1.64 2.19 
6 14.87*** -- - -- 

*p <.05 

**p < .006 

ghd 8 < .001 
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Figure 1: Graph of group recognition error means across states 


Strategy Use and Interaction Style 


Coded State 0 dialogue was analyzed by univariate tests on strategy and in- 
teraction style scores (Table 4). 

Of the three heuristic strategies, Consider Alternatives was significant. 
While Average and Gifted pairs considered moves outside legal constraints, 
these pairs were more apt than their LD peers to reject such moves. On the 
other hand, LD pair dialogue was characterized by acceptance of nonlegal 
options. 

For the Checks Problem and Representation (Draw) strategies, results 
indicated that LD and non-LD pairs used rereading, listening again to the 
audiotaped problem, and sketching the problem to the same extent. How- 
ever, given the significant solution difficulty that LD pairs experienced in 
relation to non-LD pairs, this difference was not reflected in proportional 
differences in strategy use. 

In considering affect and interaction scores across groups, pairs tended 
to be task-oriented and helpful in generating ideas and questions with each 
other. The mean group ratings for interaction indicate task involvement 
among pairs which was characterized by helpful dialogues and mutual prob- 
lem solving, regardless of achievement label. 
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Table 4 
Group Means, Standard Deviations, and F Values 
for Strategy and Interaction Variables 


Variable Gifted LDH LDM Average F Value 
Consider 
Mean 2.89 2.15 2.47 2.77 4.31* 
SD .67 .39 .40 .63 
Check 
Mean 1.42 1.80 1.64 1.55 83 
SD .64 53 56 59 
Draw 
Mean 1.31 127, 1.38 1.14 53 
SD 59 .36 47 21 
Interact 
Mean 4.94 4.95 4.91 4.88 79 
SD 19 .08 17 .25 
Affect 
Mean 1.72 1.64 1.71 1.56 57 
SD 35 22 .26 .39 
*p <.009 
Discussion 


The present study applied a processing model to the investigation of prob- 
lem solving among children with different achievement backgrounds. 
Profiles of pair moves across problem states (Figure 1) indicated that earlier 
problem states were entered more frequently and were characterized by 
testing both legal and illegal alternatives. In later problem states solution 
was accomplished by a series of recursive steps which contained fewer legal 
possibilities. For all pairs, the last half of the problem space was charac- 
terized by a decline in the number of moves. All pairs, regardless of achieve- 
ment classification, followed a one-move-at-a-time approach in the first 
three states by selecting a move and, only after selection, by performing 
some sort of evaluation test on their selection. Thus, with respect to the 
basic structure of the problem solving process, learning disabled pairs were 
not unlike gifted or average achieving peers. 

Postmove evaluation determined subsequent move selection where rec- 
ognition of past moves and consideration of problem constraints affected le- 
gality of choices. It was here that results of group differences pointed to 
critical distinctions between learning disabled and achieving solvers. Gifted 
pairs, in contrast to both Average and LD pairs used more efficient post- 
move evaluation and recognition of past moves while systematically check- 
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ing problem constraints. Learning disabled pairs, on the other hand, made 
more moves back to earlier states and included more illegal, bizarre, and 
recognition error moves than non-LD pairs. The higher rate of legal moves 
by LD and Average pairs than by Gifted pairs was a result of more frequent 
return to the start of the problem in order to try again. It may be concluded 
that both Average and LD solvers were less efficient than Gifted solvers, 
where efficiency is interpreted as selection of lowest number of legal moves 
for solution. However, Average solvers were also distinguished from their 
learning disabled peers in that LD evaluation processes were characterized 
by a significantly greater proportion of illegal, bizarre, and recognition er- 
rors at earlier problem states. 

In observing Gifted and Average pairs, these groups engaged in frequent 
questioning and discussion which evaluated potential solutions against the 
goal state and against the three problem constraints. Their early use of this 
type of cognitive monitoring made unnecessary the later use of other meta- 
cognitive strategies such as rereading or sketching the problem. In the early 
stages of problem solving monitoring, LD pairs proceeded in a trial and 
error fashion. Although they used pair discussion and questioning to the 
same extent as other groups, LD pairs ignored constraints and made prema- 
ture claims for solution based on unacceptable possibilities. 

Thus it appeared that LD students were spontaneously able to initiate, 
at a minimal level, problem solving dialogue and common strategies. One 
strategy which proved more difficult for disabled learners was that of con- 
sidering alternatives systematically. This strategy reflected ability to recall 
past moves rejected or accepted and to filter these for acceptability against 
the problem constraints. LD pair dialogue reflected less effective use of 
evaluation strategies and recognition processes than non-LD dialogue. 
Therefore, a distinction should be made between LD use of metacognitive 
strategies such as rereading, pictorial representation of the problem, or 
problem solving discussion and LD difficulty in more critical processes of 
comparing, recalling, and using common sense problem knowledge. LD sol- 
vers made significantly more attempts to reconstruct problem possibilities 
outside the given constraints. This finding points to a difference in what 
Simon (1983) has called commonsense problem knowledge based on an un- 
derstanding that the problem statement is necessary and sufficient for solu- 
tion. It appeared to be a significant factor in effective problem solving for 
disabled learners. While the greater incidence of bizarre and recognition er- 
rors in LD moves was indicative of difficulty in recalling constraints ac- 
curately, the concept of necessity to adhere to problem constraints also 
appeared to be less firmly understood by LD solvers than by non-LD solvers. 
Thus the present investigation underscored that even in a general search 
task, specific task knowledge and task-related cognitive processes were crit- 
ical missing elements in LD problem solution. 

Because of their greater apparent difficulty in using cognitive processes 
of comparison and evaluation to achieve a clear understanding of the prob- 
lem, LD groups would be expected to rely on other compensatory strategies 
such as rereading and sketching the problem. Nevertheless, LD and non-LD 


93 


A. Thomas 


pairs used these strategies with similar frequency despite LD solvers’ 
greater difficulty. It would appear that LD pair monitoring did not reflect 
adaptive or flexible use of “fix-up” metacognitive strategies, although these 
were minimally available in the repertoires of LD pairs. It would seem that 
the cognitive processing model used in this study was effective in more 
clearly delineating the nature of individual differences in problem solving. 
The relationship between cognitive and metacognitive processes in the 
problem solving of different groups of learners was further clarified based 
on the move-by-move requirements of the task presented and the spon- 
taneous dialogue generated by the pairs procedure. 

In the present study, helplessness was not a discriminating factor for LD 
groups, as no differential patterns emerged between the LD mastery and 
LD helpless groups. The finding of no differences on dimensions of negative 
affect and performance evaluation supported the hypothesis that a non- 
evaluative task environment which stressed cooperative problem solving 
minimized the negative influence of noncognitive factors on the problem 
solving of learning disabled students. Nevertheless, methodological limita- 
tions in the present study suggest caution in such conclusions. While help- 
less and mastery oriented LD students were identified based on the IAR, LD 
teacher ratings indicated similar judgments for both groups. Therefore, it is 
possible that both LD groups were similar on crucial aspects of helplessness 
despite their differential self-ratings. A recommendation for future research 
would include pretesting on challenge tasks to obtain behavioral indices of 
helplessness to supplement rating scales. 

Implications for future research center on investigation of learner no- 
tions about problem solving and applications of the problem solving pair 
technique to other areas of problem solving. In much the same way that 
studies of metamemory have uncovered idiosyncratic understandings 
among disabled learners, relationships among notions about the problem 
solving process, failure experiences, and problem solving effectiveness await 
clarification. In a similar vein, the pair procedure as a way to generate task- 
related discussion and ideas in active communication is an area worthy of 
future research consideration. In the present study all solver groups seemed 
able to respond to the presence of a known peer with activities of listening, 
mutual sharing of ideas, and cooperative planning. From this perspective 
the problem solving activity and persistence levels of LD students were con- 
siderable, compared with the typical view of the LD learner as passive and 
dependent on teacher direction. It seems important to incorporate the 
potential inherent in cooperative peer arrangements in order to more ac- 
curately assess the problem solving capabilities of students and reflect the 
real-world setting in which much actual problem solving occurs. More re- 
cently the importance of peer dialogue in developing metacognitive skills 
has been recognized (Gelzheiser & Shepherd, 1986) and training programs 
have begun to incorporate the peer component in order to increase program 
effectiveness. 
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Note 


1. Acomplete description of coding procedures and scoring criteria is available from Adele 
Thomas, College of Education, Brock University, St. Catharines, Ontario, L2S 3Al. 
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The present study investigated the relationship between program organizational features 
and student perceptions of their learning environments in secondary-level alternate school 
programs. One hundred and three students enrolled in seven secondary-level alternate 
school programs responded to the Learning Environment Inventory which measures stu- 
dent perceptions of their educational environments, while program teachers completed the 
Classroom Characteristics Rating Scale which measures: (a) differentiation of instruction- 
al tasks among students, (6) student autonomy, (c) grouping, and (d) evaluation practices. 
A number of significant relationships—both desirable and undesirable—among specific 
program organizational features and student perception variables were identified. Dif- 
ferences regarding teachers’ practices across the four organizational and instructional 
areas were also noted. Implications of these findings for teachers, curriculum designers, 
and other educational personnel are discussed. 


Alternate education at the secondary level in Canada has been charac- 
terized by much organizational and curricular variation. This variation has 
been most noticeable among alternate school programs that specialize in 
providing instruction for students with socioemotional problems (Csapo & 
Gittens, 1979). One of the primary goals of secondary-level alternate educa- 
tion has been to provide a learning environment where students who have 
experienced academic failure in the traditional school system may enjoy 
academic as well as social and emotional growth (Csapo & Stevens, 1982; 
Stevens, 1985). In order to facilitate growth in these areas, educators re- 
sponsible for planning alternate school curricula in a particular school dis- 
trict frequently design a program they believe will best meet the unique 
needs of their specific student population. Typically, this practice results in 
little program consistency from one alternate school to the next with re- 
spect to such areas as student autonomy, organization of instruction, and 
evaluation procedures. 

In recent years, researchers have begun to acknowledge the importance 
of organizational features in educational settings and their influence on 
teacher and student perceptions of school-related events (Johnson, 1987). 
To date, however, most studies involving student perceptions of their learn- 
ing environments have tended to focus almost exclusively on perceptual 
data and have ignored the importance of measuring the organizational fea- 
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tures of these environments (Chavez, 1984; Fraser, 1985). Rather, organi- 
zational differences among programs have largely been assumed. Moreover, 
studies concerning teacher organizational and instructional practices in 
various types of school programs have focused on shared perceptions of 
ability (Rosenholtz & Rosenholtz, 1981; Rosenholtz & Wilson, 1980), aca- 
demic and social power (Marx, 1985; Rosenholtz, 1981/1982), and academic 
self-concept (Simpson, 1981), rather than on student perceptions of class- 
room climate. Two notable exceptions are studies by Stevens (1985) and 
Gold and Mann (1986). Stevens examined the relationship between alter- 
nate school student perceptions of school climate and the organizational 
features that distinguished their schools. Although Stevens found that stu- 
dent perceptions of their learning environments were related to their alter- 
nate school programs’ organizational structure, the program classification 
procedure used did not address teacher organizational and instructional 
practices per se. In Stevens’ (1985) study, program organizational structure 
was more broadly construed by encompassing school goals, class size, source 
of funding, and target student population variables. Recent studies examin- 
ing the relationship between student perceptions of school rules and pro- 
gram organizational features in conventional settings have tended to clas- 
sify schools in a similar fashion by using broader categories such as size of 
school, type of school board, and number of student clubs or organizations 
rather than according to teachers’ specific organizational and instructional 
practices (Macdonell & Martin, 1986). 

In a similar study, Gold and Mann (1986) examined student perceptions 
of flexibility of school rules, commitment to the role of the student, and at- 
titude toward school in alternate and conventional school environments at 
three different intervals across a 14-month period. Results indicated that al- 
ternate school students perceived their school programs as possessing great- 
er rule flexibility, were more committed to meeting specific school expec- 
tations, and held more positive attitudes toward school than did their con- 
ventional counterparts. However, the “attitude toward school” variable in 
this study consisted of questions addressing four general components: global 
expression of liking of school, positive feelings for teachers, participation in 
extracurricular activities, and one’s mood at the end of a school day. Fur- 
thermore, the main focus of this study was the comparative effects of alter- 
nate and conventional school environments on student perceptions or at- 
titudes toward school, rather than the relationship between these percep- 
tions and the organizational and instructional features of different alter- 
nate schools. 

The purpose of this study was to investigate the link between student 
perceptions of their learning environments and the type of alternate school 
program in which students were enrolled. Two questions guided this in- 


quiry: 
1. Do alternate school programs differ in terms of their organizational and 
instructional characteristics? 
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2. What is the relationship between alternate school student perceptions of 
their learning environments and the organizational and instructional 
features that characterize these environments? 


Method 
Subjects 


Data were gathered from 103 students in seven secondary-level alternate 
school programs in the Upper, Middle, and Lower Fraser Valley regions of 
southwestern British Columbia. Each of the seven alternate school pro- 
grams was located in a separate building and in a different part of the 
school district from the regular junior high school to which it was attached 
administratively. In total, 71 boys and 32 girls were involved in the study. 
Students ranged from grades 7 to 10 with two students in grade 7, 25 stu- 
dents in grade 8, 49 students in Grade 9, 20 students in grade 10, and seven 
students who did not specify a particular grade level. All participating stu- 
dents were required to have attended their particular alternate school pro- 
gram for at least eight months prior to the beginning of the study. Program 
organizational data were also gathered from teachers at each of the seven 
schools. Only secondary-level alternate school programs which offered an 
academic core curriculum consisting of English, mathematics, science, and 
social studies were selected for inclusion in the study. General information 
regarding school curricula was furnished by teacher verbal reports of their 
instructional practices, as well as through policy and procedures manuals 
describing each of the school programs. 


Instruments 


Student perception measures. Student perceptions of their alternate 
school climate were measured by the Learning Environment Inventory 
(LED (Fraser, Anderson, & Walberg, 1982) which consists of 15 perception 
variables: cohesiveness, diversity, formality, speed, material environment, 
friction, goal direction, favoritism, difficulty, apathy, democracy, cliqueness, 
satisfaction, disorganization, and competitiveness. Alpha coefficients for in- 
dividuals range from .54 for diversity to .86 for goal direction, while test- 
retest reliability for individuals ranges from .43 to diversity to .73 for 
friction. Intraclass correlations for groups range from .31 for diversity to 
.92 for formality and disorganization. A number of excellent reviews regard- 
ing the link between student perceptions as measured by the LEI and var- 
ious learning outcomes are provided by Chavez (1984) and Fraser (1985). 
Studies involving the use of the LEI subscales as criterion variables have 
also been undertaken to measure the effectiveness of curricular innovations 
(Anderson, Walberg, & Welch, 1969; Fraser, 1978, 1979; Levin, 1980; 
Tisher & Power, 1976, 1978; Welch & Walberg, 1972), differences in stu- 
dent perceptions across various grade levels and subject areas (Anderson, 
1971; Welch, 1979) as well as across different types of school programs 
(Randhawa & Hunt, 1981; Randhawa & Michayluk, 1975; Sharan & 
Yaakobi, 1981). 

Program organization measures. Information regarding alternate 
schoolteachers’ organizational and instructional practices were gathered 
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through an instrument called the Classroom Characteristics Rating Scale 
(CCRS). This instrument is essentially a modified version of an interview/ 
questionnaire developed originally by Rosenholtz and Rosenholtz (1981) to 
measure dimensionality characteristics of classrooms across four areas: dif- 
ferentiation of task structure, student autonomy, grouping practices, and 
student evaluation. Teachers’ organizational and instructional practices are 
viewed as either unidimensional or multidimensional in nature. Students in 
unidimensional classrooms are (a) instructed to use similar materials across 
subjects, (b) grouped collectively for the purposes of instruction, (c) rarely 
permitted to make decisions regarding type or length of activity, and (d) fre- 
quently evaluated on the basis of comparing one student’s progress to that 
of another. Students in multidimensional classrooms, on the other hand, (a) 
use different materials across subject areas, (b) receive individual instruc- 
tion, (c) are permitted a greater range of instructional choices or options, 
and (d) are evaluated on the basis of individual progress. 

For the purposes of this study, the interview portion of the instrument 
was omitted, and the questionnaire portion modified to suit alternate school 
programs. The modification consisted of inserting the word “alternate” 
before the word “classroom” for the appropriate items. Teachers simply re- 
sponded by checking never, seldom, occasionally, frequently, or always to 
items asking about their instructional practices across the four subject areas 
of English, mathematics, science, and social studies. Currently, no norma- 
tive data or data regarding reliability and validity are available for either 
form of the CCRS. However, a number of studies have used the original in- 
strument to explore the relationship between classroom organization char- 
acteristics and shared perceptions of ability (Rosenholtz & Rosenholtz, 
1981; Rosenholtz & Wilson, 1980), academic and social power (Marx, 1985; 
Rosenholtz, 1981/1982), academic self-concept (Simpson, 1981), and at- 
titude toward school (Rosenholtz & Simpson, 1984). While prior use of the 
instrument as evidenced by these studies illustrates the credibility of the in- 
strument, as well as its ability to discriminate among organizational fea- 
tures of classrooms, the absence of data regarding validity or reliability is 
acknowledged as problematic with respect to the results reported in this 
study. 


Procedure 

Students completed the LEI in small group testing sessions in June. One 
morning was required to complete the data collection for four schools, while 
the remaining three schools required two consecutive mornings due to ab- 
senteeism. Groups consisting of a maximum of five students each were ad- 
ministered the inventory at each school. In a school where 12 students were 
enrolled, three groups of four students were tested. Information regarding 
student sex and grade was gathered. No other identifying information was 
requested. 

Teachers completed the CCRS individually in June. Only one score for 
each of the four dimensions per subject area was required from each alter- 
nate school program. In schools where two or more teachers were employed, 
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consensus had to be reached concerning the instructional practices for each 
dimension before a single response for each item was recorded. 


Results 


Teacher Organizational and Instructional Differences 


Results of the CCRS are presented in Table 1. Theoretically, the lower an 
individual school’s score, the higher its degree of unidimensionality for the 
organizational dimension under consideration. Conversely, as the individual 
program scores increase across dimensions, the degree of multidimen- 
sionality also increases. Since it was possible to earn a total of 5 points 
across six subject areas for each dimension, a maximum of 30 points and a 
minimum of 6 points for each of the four dimensions was possible. The table 
indicates how each of the seven schools compared with respect to the four 
program dimensions. 

Task structure. The program scores are uniformly low and quite similar 
along this dimension. The difference between the most unidimensional and 
multidimensional schools is 7 points (Program One with 9 points and 
Programs Three and Six with 16 points each). For this dimension, teachers 
were asked to respond to questions concerning how often their students use 
the same materials, such as texts, workbooks, and teacher-made materials 
across different subject areas. In general, instructional practices of teachers 
representing each of the alternate programs appear to be similar with 
regard to task differentiation. Furthermore, these practices tend to be uni- 
dimensional. 

Student autonomy. Unlike task structure, there was much variability 
among program scores regarding student autonomy or decision making 
power. For this dimension, teachers were asked how frequently their stu- 
dents were permitted to make their own decisions regarding type and 


Table 1 
Results of the Classroom Characteristics Rating Scale 


Program Organization Characteristics 


Programs Task Student Instructional Teacher 
N=7 Structure Autonomy Organization Evaluation 
1 9 8 10 18 
2 11 10 11 22 
3 12 15 16 30 
4 13 17 21 24 
5 16 20 17, 26 
6 16 18 25 30 
7 12 25 28 29 


Possible range of scores for each dimension is 6-30. Lower scores indicate unidimensionality; 
higher scores indicate multidimensionality. 
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length of activity for each subject. Teacher responses along this dimension 
ranged from never (two programs) to frequently (one program), with four of 
the seven teachers reporting that they occasionally permit their students to 
make these types of decisions. 

Student grouping. Similar to autonomy, there was much variability 
among teacher grouping practices. For this dimension, teachers were asked 
how often they group the entire class together for instructional purposes. 
Again, individual school responses ranged from never (one program) to fre- 
quently (two programs) along the unidimensional/multidimensional con- 
tinuum. 

Teacher evaluation. Relative to the preceding dimensions, scores for all 
seven programs regarding teacher evaluation were uniformly high. For this 
dimension, teachers were asked how frequently they made comparative 
evaluations when assessing individual student progress across subject areas. 
None of the responses indicated a strong unidimensional preference. Teach- 
er responses ranged from occasionally (one program) to never (three pro- 
grams). The remaining three schools indicated that they seldom made 
comparative student evaluations. 


Program Organization and Student Perceptions 


The relationships among student perceptions and program organization 
data were explored to determine the extent to which teacher organizational 
and instructional practices relate to student perceptions of their alternate 
school programs. Program means and standard deviations for each of the 
student perception variables are presented in Table 2. Pearson correlation 
coefficients were computed for each of the four program organizational 
variables including a total value across each of the 15 climate variables. The 
program raw score on the respective organizational variable was correlated 
with the 15 mean student perception scores. A number of statistically sig- 
nificant correlations emerged from these analyses and are illustrated in 
Table 3. 

Task structure. As mentioned, teachers tended to be unidimensional in 
this area by generally using similar materials for different students across 
subject areas. However, as these teacher practices become more multi- 
dimensional, significant correlations indicate that students like their ma- 
terial environment less (—.74) and perceive their programs as less formal 
(_.86), less goal-directed (—.67), and more disorganized (.81). 

Autonomy. Differences regarding the amount of autonomy or decision 
making power program teachers typically give their students has been 
noted. As student decision making power increased, student perceived levels 
of formality (-.81), speed (-.85), difficulty (-.81), competitiveness (-.93), 
cohesiveness (—.68), diversity (-.67), and goal direction (—.68) decreased to a 
significant level, while perceived level of democracy (.91) increased sig- 
nificantly. 

Grouping practices. Teacher grouping practices were widely dispersed 
along the unidimensional/multidimensional continuum. Results indicate 
that as instruction became more individualized, student perceived levels of 
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Table 2 
Program Means and Standard Deviations for the 
15 Student Perception Variables of the Learning Environment Inventory 


ear aa DUDE nan TEN 


Perception Variable Program 


1 2 3 4 5 6 7 


ee 


Cohesiveness X 21.73 23.92 22.20 22.77 20.85 20.50 20.39 


SD 2.15 2.61 3.08 2.20 2.64 2.95 3.26 

Diversity Moss). 21.02 e" 20 70 8 old eee Od mec 20.78 
SD 2.61 2.02 1.56 2.21 2.04 1.20 2.93 

Formality X 25.33 25.25 21.00 21.00 19.73 19.60 21.04 
SD 72.02 1.82 1.94 2.51 2.75 2.01 2.92 

Speed X .. 1840 . 17.08.. 16.20. ~» 13,624 1540, .915:35 14.15 
SD 4.08 2.46 1.93 2.63 2.75 4.27 3.48 

Material X 21.60 23.08 2190 2092 2055 19.40 20.96 
Environment SD 1.92 2a Tot 3.28 2.68 3.24 2.79 
Friction X% 20:73. 19.33) 94.19.1040 20.23 cco ames 00 21.74 
SD 2.43 3.14 2.33 2.59 2.46 2.40 4.41 

Goal x Mos t0 M24issv uM rIgsOn 19ST eae e000 Bam t910 20.48 
Direction SD 3.43 2.15 2.49 2.75 2.28 3.35 2.39 
Favoritism KX °""NG10) 12167 16.40 0 15,0Gae ec coup mel 7:70 15.96 
SD 3.49 3.06 2.59 3.20 3.16 2.87 2.99 

Difficulty X% $2093 1950 w16.100 16,15 seel7 500 17.10 15.96 
SD 1.28 1.73 2.33 2.73 3.12 3.14 2.47 

Apathy X 1693 1242 1590 1662 1640 21.00 17.76 
SD 2.60 3.03 3.28 3.78 3.17 2.62 3.73 

Democracy yA 12.80 14.83 15.90 17.69 16.05 16.50 19.83 
SD 1.37 1.53 2.88 1.75 2.26 2.55 3.33 

Cliqueness X 1940 1817 1880 1808 20.00 20.40 19.00 
SDmeee ag 1.95 2.53 1.85 3.13 3.20 2.63 

Satisfaction X.coh7i404) 219.83" 1 S;70 Ie S4ee eoswen 15:00 17.70 
SDueus 2.62 2.21 313 1.82 2.00 3.42 

Disorganization Xx 12.87 11.92 16.00 14.38 16.65 16.65 15.09 
SD apeced 2.23 3.13 2.43 4.15 2.56 3.50 

Competition X 21.27 20.75 15.80 15.00 16.20 15.60 13.04 
SD 2.02 1.54 3.16 2.97 3.78 2.99 2.01 


friction (.70), apathy (.68), and democracy (.90) increased significantly, 
while levels of perceived formality (-.75), speed (-.82), difficulty (-.79), com- 
petitiveness (-.91), attractiveness of the material environment (-.70), and 
goal direction (—.75) decreased significantly. 

Student evaluations. Scores along this variable were more uniformly 
multidimensional than they were for the other three program organization 
variables. As multidimensionality increased, levels of student perceived for- 
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Table 3 
Correlations Between Student Perceptions of their 
Alternate School Programs and Teacher Organizational 
and Instructional Practices (N =7) 


Program Organization Areas 


Student Task Student Student Teacher 
Perceptions Structures Autonomy Grouping Evaluations Total 


Cohesiveness -.49 —.68* —.65 -.50 —.67* 
Diversity —.35 ~.67* -.48 -.64 —.62 
Formality —.86** —.81** —.75* —.83* —.89** 
Speed —.60 ~.85** —.82** -.59 —.83** 
Material Env. —.74* —.59 —.70* —.49 —.70* 
Friction 33 52 Whi 25 4518) 
Goal Direction -.67* —.68* —.75* —.75* —.80** 
Favoritism .64 47 .38 .46 51 
Difficulty 0G —.81** —.79* —.85** —.86** 
Apathy 50 47 .68* 47 61 
Democracy 40 .91** .90** .68* oo" 
Cliqueness sere .25 vas .30 34 
Satisfaction —.47 —.28 —.52 —.34 —.45 
Disorganization  .81** .69* 61 5 a ée}e 


Competitiveness -.56 —.93** —.91** —.82** —.94** 
7D = Oo =..01 


mality (—.83), difficulty (-.85), competitiveness (-.82), and goal direction 
(_.75) decreased significantly, while perceived levels of disorganization (.81) 
and democracy (.68) increased significantly. 

Total. When data from all four program organization variables were con- 
sidered collectively, a number of significant relationships—both desirable 
and undesirable—emerged. Specifically, as programs became more multi- 
dimensional across all four organizational areas, students perceived their 
programs as less formal (—.89), less rushed (-.83), less difficult (-.86), less 
competitive (-.94), and more democratic (.88). Conversely, they also per- 
ceived their alternate school environments as less physically attractive 
(-.70), less cohesive (—.67), less goal-directed (—.80), and more disorganized 
(19)s 
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Discussion 

The findings of the present study indicate that teachers, administrators, 
and other educational personnel responsible for planning alternate school 
curricula should review the organizational and instructional features of al- 
ternate school programs if they wish to create learning environments that 
are perceived positively by their students. As noted, students perceived their 
programs as informal, comfortably paced, democratic, not difficult, and 
noncompetitive as multidimensionality increased across all organizational 
areas. Less encouraging were results indicating that students also perceived 
their learning environments as uncohesive, physically unattractive, lacking 
goal direction, and disorganized. These findings suggest that multidimen- 
sional instructional practices among alternate school teachers have aided in 
creating learning environments that are perceived as fair in most respects 
by their students. However, by creating these types of environments, it ap- 
pears that teachers have sacrificed other equally important characteristics 
of their alternate school programs such as clearly established program 
goals, order or organization, and feelings of belonging or affiliation as per- 
ceived by their students. 

To illustrate, it was discovered that student level of perceived cohesive- 
ness, diversity, and goal direction is negatively correlated with greater mul- 
tidimensionality across the organizational variable of student autonomy, 
while level of perceived disorganization is positively correlated with this 
same variable. These findings suggest that satisfaction among alternate 
school students decreases as opportunities to make individual curricular 
decisions increase. In other words, greater student autonomy regarding the 
selection of particular subjects or tasks is not necessarily associated with 
positive student perceptions of their alternate school programs. This finding 
is surprising because curricular inflexibility within the conventional school 
has frequently been cited as a major reason among alternate school stu- 
dents for dissatisfaction with this environment as well as their inability to 
cope with its demands (Gardener & Farnsworth, 1977; Gold & Mann, 
1986). More importantly, however, it suggests that a more unidimensional 
or teacher-centered approach to instructional decision making may be pref- 
erable to an individualistic approach, especially in terms of enhancing stu- 
dent perceptions of their alternate school programs. By taking responsibility 
for deciding which activities students must perform, and by setting con- 
straints on performance (e.g., number of exercises to be completed, amount 
of time per activity, etc.), teachers may have some positive effect on the per- 
ceptions alternate school students hold about their school programs. 

It is also interesting to note that student perceived levels of friction and 
apathy are positively correlated with greater multidimensionality across the 
variable of student grouping, while perceptions of goal direction are nega- 
tively correlated with this same variable. In other words, as student group- 
ing practices become less collective and more individualized, student per- 
ceptions of friction and apathy in alternate school programs increase rather 
than decrease. Perhaps more importantly, student sense of confusion re- 
garding the actual purposes or goals of their alternate school programs also 
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increases as instruction becomes more individualized. These findings tend 
to corroborate existing data concerning the effects of individualized instruc- 
tion which provides only marginal support for its usefulness in promoting 
either prosocial gains or more favorable affective responses toward school- 
ing (Good & Brophy, 1986; Peterson, 1979). These findings seem especially 
important for alternate school teachers. In our discussions with the teachers 
who participated in this study, we discovered that many of them provided 
opportunities for individualized instruction simply because they felt that the 
class-as-group instructional approach typically found in conventional 
schools might have been perceived as too competitive, threatening, and im- 
personal by their students. Moreover, given the small number of students in 
their school programs, many of these teachers saw individualized instruc- 
tion as not only a preferable alternative, but a manageable one as well. 
However, while little direct research regarding the comparative effects of 
individualized versus group instruction in secondary-level alternate school 
programs currently exists, the results of the present study as well as those 
described by Good and Brophy (1986) suggest that a more unidimensional 
approach toward grouping students for the purposes of instruction by 
teachers seems warranted. 

To summarize, the results of the present study point to the need for al- 
ternate school personnel to become more sensitive to the relationship that 
exists between teacher organizational and instructional practices and stu- 
dent attitudes or perceptions of their programs. If teachers hope to create 
alternate school programs where social climates are perceived as differen- 
tially more supportive by students than traditional school climates, then a 
more unidimensional approach to certain features of program organization 
is necessary. Further research aimed at identifying more specifically those 
organizational and instructional practices that are linked to positive percep- 
tions of classroom climate is needed to enable program personnel to modify 
and improve existing alternate school curricula. 


Note 


1. Further information about each of the seven alternate school programs may be obtained 
from John Woudzia, Faculty of Education, Simon Fraser University, Burnaby, British 
Columbia, V5A 1586. 
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BOOK REVIEW 


EDUCATION, RESEARCH, INFORMATION SYSTEMS AND THE 
NORTH. By W. Peter Adams (Ed.). Ottawa: Association of Canadian 
Universities for Northern Studies, 1987, 362 pp. 


This is an excellent resource for students, teachers, researchers, and others 
interested in the Canadian North. Developed largely from the proceedings 
of a meeting of the Association of Canadian Universities for Northern 
Studies (ACUNS) held in Yellowknife in April 1986, the compilation focuses 
on five major areas of northern interest: education and research, higher ed- 
ucation and training, teacher education, distance education, and informa- 
tion systems. The work also includes an extensive section on the northern 
research, teaching, and service interests of staff of the Association’s 35 
member universities. 

The editor is to be commended for compiling and sorting such a wealth 
of information. Given the work’s overall editorial excellence, one hesitates 
to find fault with what has been done; however, there are two areas that 
deserve some attention. First, Adams might have taken time in the intro- 
duction to delineate the various regions that are generally classified as 
being part of the Canadian North. Hamelin’s northern categories are 
referred to in one article, and Finkler provides a definition that reflects an 
administrative research focus, but the term North remains undefined 
throughout the book. This raises a second question: Why were there con- 
tributions from some northern areas and not from others? The territories 
receive considerable attention, and there are valuable discussions on 
programs in Labrador and Arctic Quebec, but little is included on the north- 
ern areas of the other provinces. It would have been helpful, for example, to 
have had reports on the activities of the Northern Alberta Development 
Council and its counterparts in Saskatchewan and Manitoba. That not 
much is said in the work about provincial northern programs suggests that 
ACUNS operates with greater ease in areas where federal interests and 
responsibilities are paramount. Provincial commentaries, particularly those 
from the West, give scant attention to their own northern areas and instead 
devote much of their comment to affirming a willingness to serve areas 
above the 60th parallel. There is clearly a need for more collaborative re- 
search. And while ACUNS has undoubtedly attempted to moderate compet- 
ing jurisdictional interests to effect coordinated and long-term research 
strategies, it is unfortunate that issues relating to the state of federal- 
provincial cooperation in northern matters are not addressed anywhere in 
the work. 

The section on education and research provides an overview of services 
available to northern secondary students as well as of a number of special 
research and training programs sponsored by such agencies as the Yukon 
Science Institute. There is a consensus throughout that the elementary and 
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middle school programs provided for aboriginal children in the Territories 
are ineffective. Implicit in the reports of such hiring agencies as the Depart- 
ments of Public Works and Renewable Resources is the assumption that 
until aboriginal children attain matriculation or equivalent status, they will 
not be able to participate in the sophisticated technical and scientific ac- 
tivities commonly associated with most northern ventures. The frustration 
experienced in participating in a failing educational system is exemplified in 
Tourangeau’s comments on the needs of Métis children; in Higgins’ discus- 
sion on the educational activities of the Inuit Tapirisat; and in Duggan, 
Johnson, and Naidoo’s article on secondary school programs in the 
Northwest Territories. Some agencies, such as the Northern Heritage 
Society in Yellowknife, offer training programs for older Native children. 
But because the numbers involved are small and the programs appear to be 
continually on the brink of financial collapse, their overall benefit is mini- 
mal. Of much greater importance than references to such ventures as the 
Heritage project is that nothing substantial is suggested in the education 
section on how the school systems might be improved. Surely more is re- 
quired than general affirmations about the need for an efficient school sys- 
tem. 

The research presentations largely reflect developmental or applied re- 
search operations. While valuable in themselves, they have little to do with 
formal educational programs at the pre-university level. Indeed, the only re- 
search paper that deals with in-school programs is Stairs and Annahatak’s 
article on Primary Inuktitut programs in the section on teacher education. 
Educational and other forms of human science research are clearly not a 
priority for either the universities or the federal government. As Finkler 
points out, less than 30% of the budget of the federal government’s north- 
ern scientific training program was devoted to human science research by 
the universities during the period 1980-1985. It is not clear how much of 
this allocation was in support of school based research, but the presenta- 
tions and other comments suggest that this area receives very little funding. 

The sections on higher education and teacher education are more 
promising both in terms of program focus and outcomes. The articles by 
Stapleton and Cleveland describe the development of the Northwest 
Territories’ Arctic College. This institution, with its several campuses, is 
providing a good range of programs despite such problems as staffing, 
course development, the readiness of applicants, and the vastness of the 
area served. There are frequent calls in the papers for interinstitutional 
cooperation. And while the matter of institutional autonomy is mentioned 
as a deterrent in establishing “links between Arctic College and the univer- 
sities and colleges of the South” (p. 89), an equally important but unstated 
concern has to do with how the cost of such cooperation will be met. 

Arctic College programs are based to a significant extent on the success 
of several earlier territorial postschool programs—the cooperative and 
teacher training initiatives especially. Lyall’s article on Arctic cooperatives 
is somewhat disappointing because it ignores the role that missionaries 
played in promoting and supporting the movement and because it stresses 
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the general administrative rather than the product and delivery arrange- 
ments adopted by local communities. The accounts of the origins and 
growth of teacher training programs, on the other hand, are more com- 
prehensive. One might fault Cram and Wilman for being overly enthusiastic 
about the accomplishments of the Kativik and EATEP teacher training 
programs, but there is little doubt that these programs, together with the 
initial teacher education project in Yellowknife, are excellent models for all 
areas of northern higher education. 

Much current thinking about northern teacher education programs is 
based on the conviction that instruction should be carried out insofar as 
possible on a face-to-face basis at the community level. Although this deliv- 
ery mode differs somewhat from those discussed in the articles on distance 
education, none of the approaches proposed involves students leaving their 
communities for extensive periods. Apart from Owen who provides an inter- 
esting profile of a typical distance delivery learner, the section’s contrib- 
utors do not have much to say about their client’s ethnicity or social eco- 
nomic status. The articles by Hickman and Smith are particularly notewor- 
thy in that they, unlike the other commentators, at least suggest using Na- 
tive languages and local cultural orientations in determining course con- 
tent. 

The section on information systems demonstrates the vast amount of 
data that is available on the North in libraries, archives, and other sources, 
and the recently developed on-line systems that make it possible to have al- 
most instantaneous access to much of this information. The section is trou- 
blesome nonetheless because it shows that one requires a fairly high level of 
both computer competence and funding support to use these systems effec- 
tively. This suggests that Natives and other northerners will find it increas- 
ingly difficult to obtain data on their own and that they will become more 
dependent on specialists not only for information but also for the inter- 
pretative frameworks in which such information will be used. 

As stated earlier Education, Research, Information Systems and the 
North is an extremely valuable resource. From an historical perspective its 
contents indicate that a new form of northern development is in place, one 
that is much more comprehensive and far-reaching than such previous sys- 
tems as the turn-of-the-century fur trade or the coming of government in 
the 1950s. The volume depicts an extensive range of knowledge and re- 
search systems and, in keeping with the editor’s aspirations, it should be 
looked on as a basic reference for anyone interested in the North and espe- 
cially by those who live and work in all parts of northern Canada. One can 
only hope that this source book will be the first of many such ACUNS publi- 
cations. 


Robert Carney 
University of Alberta 
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THE EDITOR’S PAGE 


Educational Research—Indigenous or Exotic? 


In early December 1988, I attended the annual conference of the Australian 
Association for Research in Education. The theme of the conference was the 
title for this editorial. Like us, the Australian educational research com- 
munity is strongly influenced by American ideas. Many of the researchers 
were trained in the United States; the journals of the American educational 
research community are viewed as being more desirable outlets than na- 
tional journals; and many of the dominant research themes are those that 
are on the American agenda. The question that was raised in a few of the 
sessions was what, if anything, should be done about it? 

According to Richard White of Monash University, the influence is uni- 
directional. After examining many of the American research journals, White 
concluded that American authors seldom cite foreign published material, 
and when they do it is almost always a book rather than a journal. White 
correctly notes that the best chance that a researcher has of having an in- 
ternational influence on educational research is to publish in American or 
British journals. But what he doesn’t answer is, “What effect does this have 
on the development of an indigenous approach to educational research?” Is 
there a case to be made for the development of a unique perspective on 
problems? I think that there is. 

In my own field of educational measurement, many of the topics that ap- 
pear in the American literature are driven by the social-legal context of 
American education. For example, minimum competency testing, certifica- 
tion testing for teachers, and item bias are all topics of major interest that 
have their roots in American legislation. There are examples from other 
fields—mainstreaming in special education comes immediately to mind. 
While it would be inaccurate to claim that all research published in 
American educational research journals must fit their national framework, 
the possibility of having a large audience for your work is a powerful rein- 
forcer, and I suspect that some of our colleagues are influenced by it, not 
only in what they research, but by how they research it. 

It is important that we encourage research from our own perspective, 
not only research on problems for which there is a Canadian concern, such 
as effects of French immersion, but also research on the generic problems. 
It may seem self-serving to ask you to make AJEFR your outlet of first 
choice, but examine the articles in this issue and see if you don’t agree that 
they not only make a strong contribution to educational thought, but they 
also have a distinctive approach to them. To underline the point I have 
placed George Buck’s paper on M.E. LaZerte as the lead article for this 
issue. LaZerte was a man who exemplified the kind of researcher that our 


community needs. 
T.O.M. 
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M.E. LaZerte: 
Pioneer Educational Innovator 


M.E. LaZerte was a prominent educator in Alberta during the first half of this century. 
This article describes LaZerte’s learning theories and several instructional devices he 
created and tested. Their merits and weaknesses are discussed from LaZerte’s viewpoint, 
as well as their relationship to general instructional theory and pedagogy. The article also 
compares LaZerte’s innovations and learning theories to other contributors, including 
those of the prominent American psychologist B.F. Skinner. LaZerte’s findings from his 
instructional devices may have relevance in assessing the efficacy of microcomputers used 
for instruction. 


It has been over 35 years since Milton Ezra LaZerte (1885-1975) held an of- 
ficial position in the University of Alberta. Why should his name be remem- 
bered and why should anyone still consider the merits and the findings of 
his research? Although LaZerte began his career in education in rural Alber- 
ta, he did not conform to the common yet erroneous notion that most rural 
teachers, especially those in Western Canada, possessed pedagogical skills 
inferior to those of teachers in the large centers of central and eastern 
Canada and the United States. He not only excelled as a teacher, he also 
demonstrated his skills in both educational innovation and administration 
early in his career (Chalmers, 1978). 

LaZerte was the first director of the School of Education at the Univer- 
sity of Alberta during the 1930s, and he later became the first Dean of that 
university’s Faculty of Education; this was the first faculty in any provincial 
Canadian university that prepared all teachers for that province (Johns, 
1981, pp. 192-193). Immediately following World War I and well before ob- 
taining responsible administrative positions, LaZerte addressed the prob- 
lems encountered by many teachers attempting to teach difficult concepts 
to large and often ungraded classes. A believer in scientific research meth- 
ods as well as innovative experimentation, LaZerte realized that many of 
the problems experienced by teachers as well as students might be al- 
leviated in part by instructional materials designed to be used by an in- 
dividual student without constant supervision (LaZerte, 1922, p. 30). It is 
significant to note that LaZerte began his work in this area several years 
before the so-called “progressive” movement took root in North American 
education (Charyk, 1974, pp. 277-278; LaZerte, 1945, p. 12). As early as 
1922, LaZerte noted a need for such instructional materials: 
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There is little apparatus in our schools to assist the teacher in giving a practical set- 
ting to the work that covers the field from group counting in primary years to the 
development of the angle-sum theorem in grade VIII geometry. (p. 30) 


During the past 30 years, many individuals and groups from various dis- 
ciplines, particularly outside education, have attempted either to “improve” 
or to “save” North American education by means of various teaching and 
testing machines. Certain American psychologists made the claim in the 
mid-1950s that no educators had even considered, much less experimented 
with, automated teaching (Skinner, 1954). Clearly they were unaware of 
the work and publications of LaZerte. The behaviorist B.F. Skinner devel- 
oped what he called “teaching machines” that he felt would release in- 
dividual teachers from boring and onerous tasks and enable them to spend 
their time doing more important duties. Skinner also claimed that his 
devices produced a significantly higher level of achievement among his stu- 
dents at Harvard University (Skinner, 1983, pp. 139-140). He could not un- 
derstand most educators’ reluctance to commit themselves to the use of 
either his machines or other teaching machines (Skinner, 1984). Another 
psychologist, Sidney L. Pressey, claimed that he had invented the first 
teaching machines during the early 1920s and that no interest was shown 
in them by educators, even though, as he argued, they provided “great 
savings in material and labor” (Pressey, 1932, pp. 671-672). 

For a brief period during the 1960s, interest in teaching machines 
seemed to wax among many educators. Consequently, several companies 
designed and produced a variety of devices (Hendershot, 1964). In practice, 
however, teaching machines did not achieve the goals which Skinner had 
imagined, nor did such machines see widespread use, especially in Canada 
(Rutherford, 1961). No one at that time seemed to know that M.E. LaZerte 
had experimented with and rejected the use of auto-instructional devices of 
his own design, using primary and secondary school children as his ex- 
perimental subjects, at least 30 years before Skinner came on the scene to 
promote an interest in teaching machines (Charyk, 1974, pp. 277-278; La- 
Zerte, 1933, p. 5). If LaZerte actually tried out teaching devices, what were 
his systems and the machines themselves like? Why was no attempt made 
to have them mass-produced and used in the schools in order to improve 
pedagogy and student learning? And how are they relevant to our age of 
microcomputers? The answers to these questions can be found by studying 
LaZerte’s experiments and the machines themselves, by considering his 
findings, and by taking into account the mood and educational environment 
of the time. 


Background 
LaZerte received a B.Educ. degree (a graduate degree of the time, similar to 
the contemporary Master of Education degree) from the University of Al- 
berta in 1926. His thesis, A Study of the Methods Used by Elementary 
School Pupils in Solving Problems in Arithmetic, contains some interesting 
conclusions related to learning theory. LaZerte (1926) stated that, “the abil- 
ity of pupils to find correct answers to problems in arithmetic is a poor 
measure of their actual problem-solving ability” (p. 102). He also found that 
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“much problem-solving is but a slavish adherence to type methods of proce- 
dure which become associated with certain language and number situations 
that exist in the presented problem” (LaZerte, 1926, p. 102). He took the 
findings as indications that the pedagogical methods employed by many 
teachers were unsatisfactory because the action of ascertaining a correct 
answer to a mathematical problem does not indicate to the teacher whether 
the underlying problem-solving strategy or concept has been either learned 
or understood by the student. These findings were, of course, not new. The 
Roman orator and educator, Marcus Fabius Quintilianus, known as Quin- 
tilian, had made the same observations as early as the first century A.D. 
(Quintilian, 1. 1. 24-25). It is also possible that similar observations were 
made by others prior to Quintilian. The common findings of Quintilian and 
LaZerte indicate that many educators are conservative; they believe that 
“traditional” methods, no matter how demonstrably inappropriate and in- 
ferior, are not to be displaced by newer ones. Both LaZerte and Quintilian 
also concluded that repetitive instruction such as drill and practice and 
plain memorization do not provide the student with an adequate facility for 
problem solving. This conclusion is still valid, especially in the criticism of 
modern “task analyses” (LaZerte, 1933, pp. 3-4; Quintilian, 10. 2. 2-5). It is 
of interest to note that Quintilian’s and LaZerte’s beliefs are diametrically 
opposed to those of Pressey and Skinner. Both of them advocated a be- 
havioristic approach to learning, one in which the notion of the individual 
student formulating concepts about the subject matter being taught is dis- 
missed as nonsense (Skinner, 1954). Although LaZerte acknowledged that 
his findings were based on a limited sample, he intended to investigate the 
matter further (LaZerte, 1926). 

LaZerte worked on his Ph.D. at the University of Chicago under Charles 
H. Judd, a contemporary of Edward L. Thorndike. LaZerte’s dissertation A 
Diagnosis of Difficulties Encountered in Solving Problems in Arithmetic, 
which built on his work for his B.Educ. thesis, was completed in 1927. Judd, 
unlike the behaviorists, shared LaZerte’s belief that it was necessary for a 
student to develop a concept of what was being taught, rather than simply 
being able to parrot it. Judd stated that, “while formal drill gives a superfi- 
cial mastery of arithmetic processes, it fails utterly to give pupils that 
higher training in reasoning which might be derived from an effort to un- 
derstand the meaning of number operations” (LaZerte, 1933, p. x). LaZerte 
endeavored to devise methods and apparatus, not only to evaluate students’ 
understanding of problem-solving processes, but also to develop devices that 
could instruct logical problem-solving procedures so that a student might 
develop an understanding of the underlying concepts. 


Early Innovations 
Envelope Test 
One of LaZerte’s first methods for assessing pupils’ knowledge of problem- 
solving procedures was the Envelope Test (LaZerte, 1933, p. 12). The sub- 
ject was presented with a large envelope on which was typed a question. 
Two smaller envelopes, located inside, each had a different problem-solving 
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procedure typed on the outside. The subject was to select only one of the en- 
velopes, the one that was believed to display the correct procedure. This in 
turn contained two more envelopes with further steps. After further selec- 
tion, the student would be left with one envelope, inside of which was a 
card. The card contained the last step of the problem on one side and a mes- 
sage on the other. Only one of the cards in the envelope array stated “This 
is the correct card”; the others stated, “You are wrong. Go back to the 
beginning and try again” (LaZerte, 1933, pp. 12-13). Figure 1 illustrates the 
hierarchical nature of the envelope test. 

LaZerte noted that the number of both cards and envelopes was altered 
to accommodate questions of different complexity. In addition, he stated 
that some sequences possessed intermediate blank cards or envelopes which 
indicated that the examinee was on the wrong track. This modification was 
intended to save time, serving to inform the student of an erroneous choice 
early in the procedure. Although a blank card or envelope informed the ex- 
aminee that an error had been made, it did not provide any corrective hint 
or cue (LaZerte, 1933, p. 13). In this manner, the Envelope Test did not in- 
struct the examinee, and the procedure tested only what the examinee knew 
at the time of the test. The Envelope Test possessed the capability, never- 
theless, of providing instruction in problem-solving strategy, if it was de- 
sired. LaZerte had created a rudimentary branching program anticipating 
those found in many “scrambled books” or programmed texts produced 25 
years later (Lumsdaine & Glaser, 1960). 
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Figure 1: The hierarchical envelope test devised by LaZerte 
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The Problem Cylinder 


There were distinct disadvantages to the Envelope Test. In order to prevent 
cheating, a supervisor had to be present while the test was in progress. The 
supervisor, as well, had to restore the cards and the envelopes after each 
test. Each Envelope Test, therefore, required the undivided attention of a 
supervisor who, if a different system were used, could have been supervising 
several examinations simultaneously. In addition, the relatively small num- 
ber of decisions available to the examinee meant that one could possibly 
guess the correct sequence of envelopes as well as the correct card (LaZerte, 
1933, p. 58). To alleviate these problems, LaZerte devised, and had con- 
structed in 1930, a machine which he called the “problem cylinder” (Figure 
2). Although others such as the psychologist Pressey were also developing 
testing machines at this time, there is no evidence in the literature or else- 
where to indicate that LaZerte’s work was anything but an independent en- 
deavor (personal communications with H. T. Coutts, 1987). There is no sim- 
ilarity in the concepts behind or the design of the machines or the learning 
theory implicit in its structure. 

LaZerte’s apparatus consisted of a wood and brass frame which housed 
five laminated wooden cylinders. The cylinders were constructed so that a 
keyway was present throughout their length. In addition, the center lamina- 
tions were designed so that the cylinder could be rotated around a key if it 
was positioned in the center of the cylinder. Each cylinder possesses five 
brass card holders, evenly spaced about the circumference. The card holders 
are intended to hold segments of equations or portions of a problem-solving 
strategy. In between each card holder is a small hole into which one can in- 
sert a brass pin located in a bar running along the top of the frame. The pin 
is intended to hold the particular cylinder in the selected position. The 
cylinders are mounted horizontally onto a tubular brass shaft which also 
has five steel keys fastened to it in a linear fashion. The keys are arranged 
so that they are normally located within the center of each cylinder, thus 
permitting each cylinder to rotate freely and independently. A square brass 
rod, which passes through a squared hole in the frame, is attached to one 
end of the shaft. The squared rod prevents the shaft from rotating so that 
the keys on the shaft will always be positioned along the top of the shaft. If 
all the cylinders are rotated on the shaft so that their keyways are aligned 
with the keys, the shaft can be slid one and a half inches to the right. This 
action signifies to the user that the problem presented has been solved cor- 
rectly (LaZerte, 1933, p. 59, and author’s observations). The operation of 
the problem cylinder is similar to cylindrical bicycle locks that are used at 
the present time. When the correct numbers are selected on the cylinders of 
the lock, the keyways of the cylinders are aligned, allowing the release of a 
keyed shaft within the lock, thus enabling the chain or wire locking the 
bicycle to be removed. 

A total of 3,125 arrangements of the cylinders is possible, so that the 
probability of an individual guessing the correct position of all five cylinders 
is extremely unlikely. In operation, the subject was presented with a mathe- 
matical problem and was then instructed to solve it, starting with the left- 
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most cylinder and proceeding to the right. Every cylinder held five cards, 
each of which displayed a part of a problem solving strategy. The subject 
was to select what was believed to be the correct strategy among the five 
cards on each cylinder. This was accomplished by the subject rotating each 
cylinder so that the desired information faced him or her. After each selec- 
tion was made, the subject was supposed to insert a brass pin into a hole in 
the top of the cylinder to prevent it from being moved while the next selec- 
tion was undertaken. When it was felt that all five cylinders were in the cor- 
rect position, the subject was to pull on the shaft. If the shaft did not slide to 
the right, one or more cylinders were improperly positioned, and this was an 
indication to the subject that a mistake in the steps of solving the problem 
had been made and that he or she should try again. 

LaZerte (1933) found that the problem cylinder was intriguing to stu- 
dents and was highly motivational. He stated that, “all subjects were very 
much interested in the problem presented to them. After the experiment 
was completed many requested to continue their efforts. Motivation was 
strong throughout” (p. 60). In a fashion similar to that of the Envelope 
Test, the problem cylinder could be arranged so that fewer than five cylin- 
ders could be used. In such cases, those cylinders not in use would not have 
cards inserted in their card holders. In addition, the cylinders not in use 
would be pinned in the unlocked position. 

Unlike the Envelope Test, however, the problem cylinder could be used 
without constant supervision because there was no easy way in which the 
subject could cheat or sabotage the machine. Besides being a testing device, 
the problem cylinder could be used to demonstrate logic or the logical 
analysis of a problem. The question noted at the outset of this article comes 
to mind: Why was no attempt made to have the problem cylinder mass- 
produced and used in the schools in order to improve pedagogy and student 
learning? The answer is provided by LaZerte’s findings. Although high 
motivation and interest were attributed to the problem cylinder in a kind of 
“halo” effect, LaZerte (1933) also discovered that the skill required to un- 
derstand and to manipulate the apparatus added to the difficulty of the 
problem being solved: “It is apparent that the task of solving the problems 
on the cylinder is much more difficult than when presented in the usual 
form” (p. 65). Data gathered from comparison groups verified his findings 
(LaZerte, 1933, pp. 63-67). It seems probable, therefore, that if LaZerte’s 
problem cylinder were to be used successfully as a “teaching machine,” in- 
structions and practice exercises would have to be provided so that the 
manipulation of the machine would not add to the difficulty of the problem 
presented. Thus much time and effort would have to be expended on train- 
ing in the use of the machine. Mutatis mutandis, the same criticism may be 
applied to microcomputers used for instruction. In addition to being famil- 
iar with the physical hardware such as the keyboard and peripheral devices 
such as a mouse, the computer user must also know the peculiarities of the 
programs used. It is unlikely, therefore, that the user will be able to give full 
and undivided attention to the primary task until some time is spent in 
learning about the machine and the program being used. This phenomenon 
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has been experienced by most microcomputer users, including some who 
describe the frustration caused by the problem (Vargas, 1986). Given this 
drawback, LaZerte concluded that the use of a teacher was more efficient 
and more effective than the use of his problem cylinder. 

While the problem cylinder might appear to be primitive by current tech- 
nological standards, it must be kept in mind that in 1930 the areas of in- 
structional devices and electronics were both extremely limited in their 
development. In addition, most small school boards then in existence pos- 
sessed limited funds and no electricity; it therefore seems unlikely that tech- 
nologically complex and expensive apparatus would or could have been pur- 
chased by these school boards. Had his experimental studies shown the 
problem cylinder to be a successful teaching device, or if LaZerte, like Skin- 
ner 25 years later, had decided to market the device despite indications that 
it would not be successful in the education market, the problem cylinder 
could probably have been manufactured inexpensively because of the small 
number of parts. If the brass parts were replaced by steel or aluminum, and 
the wooden parts replaced with Bakelite, the cost could have been reduced 
even further. 


Later Innovations 
Activity Oriented Mathematics 


Between the mid-1930s and the mid-1940s, LaZerte’s time was primarily 
consumed by administrative obligations, but his interest in developing test- 
ing and instructional devices did not disappear. By the end of World War II, 
LaZerte had been testing new “activity” mathematics programs in the 
University of Alberta’s demonstration school (LaZerte, 1945, p. 12). The 
programs, intended for primary grades, entailed considerable laboratory 
time for each pupil. LaZerte (1945) noted that, “the pupils spent 60% of the 
time devoted to arithmetic in the laboratory” (p. 12). In addition, the pro- 
gram was intended to be primarily individualized instruction where the stu- 
dents learned largely on their own and the teacher supervised. In this case, 
“individualized instruction” did not refer to the following of a rote program 
individually. It was expected that each student would perform the experi- 
ments outlined and discover for himself or herself the concept or concepts 
involved (LaZerte, 1951, p. 1). The laboratory work consisted of the use of 
instructional aids that were intended to facilitate the concrete representa- 
tion of concepts, as well as to facilitate problem solving. The apparatus in- 
cluded bead frames, various number boards and charts, special colored 
disks, cut-outs, and rudimentary measuring devices (LaZerte, 1945, p. 12). 
While it produced favorable results, LaZerte noted that this “activity pro- 
gram” was similar to the methods he had used prior to 1930, “before the 
progressive movement reached Alberta” (LaZerte, 1945, p. 12). This work 
culminated in the production of the combination text and activity manual 
entitled Numbers Tell Their Story, first published in 1953. 

Although the problem cylinder had been considered unsatisfactory for in- 
dividualized instruction, LaZerte continued to believe that instructional ap- 
paratus could be useful in certain situations. LaZerte (1953) stated that, 
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“the emphasis on the use of apparatus and equipment should prove of ines- 
timable value to the teacher in the rural and ungraded school” (p. ii). In 
many such schools, the teacher was responsible for several grades simul- 
taneously, so the time that the teacher could spend with each grade was ex- 
tremely limited. Although Alberta had embarked on a program of elim- 
inating one-room rural schools through centralization during World War II, 
the process took many years to complete. LaZerte’s observations continued 
to be valid for several years (Alberta Department of Education, Annual 
Reports, 1939-1965). The use of the Numbers Tell Their Story program ap- 
pears to be ideally suited for such environments. LaZerte (1953) noted that 
“if he [the student] is to profit from the course, each pupil must do all the 
work himself. The teacher must not teach demonstration lessons. It is as- 
sumed that all pupils will have some equipment in their hands while the 
class is at work” (p. ii). 

It should be recalled that LaZerte’s desire for such an individualized pro- 
gram may be observed as early as 1922 (LaZerte, 1922, p. 30). At that time, 
there were some prominent psychologists who believed that practical and 
concrete methods should be used in the instruction of concepts. Thorndike 
(1923) proposed three criteria for a successful program: 


(1) Provide enough actual experiences before asking the pupil to understand and use 
an abstract or general idea. (2) Develop such ideas gradually, not attempting to give 
complete and perfect ideas all at once. (3) Develop such ideas so far as possible from 
experiences which will be valuable to the pupil in and of themselves. (p. 179) 


LaZerte’s (1953) Numbers Tell Their Story program appears to have 
met all three of Thorndike’s criteria. To be sure, much of the material men- 
tioned above, such as bead frames and felt cut-outs, were already available 
from school suppliers. Indeed, some of the items are still available (Primary 
School Supplies, 1987). Some of the apparatus was designed by LaZerte, 
however, and was not already available. Such equipment included percep- 
tion cards (domino-like flash cards); number board (a wooden tray which 
could hold colored blocks in order to actualize addition and subtraction); 
place value frame (a shallow cardboard or wooden box marked in thirds, 
with nine slots in each third designed to hold colored disks); and the 
hundred board (a 23-inch square board, marked in quadrants with 25 hooks 
or nails arranged in rows in each quadrant, to illustrate submultiples of 
100) (LaZerte, 1953; LaZerte, Dey, & Svidal, 1959). Originally, if one 
desired, complete sets of equipment could be ordered from the publisher 
(LaZerte, 1953, p. iv). In addition, concise instructions were provided on 
how to fabricate all the equipment mentioned in the book. As LaZerte 
(1953) noted, “if a school feels that it cannot undertake the expense of one 
set per child, the laboratory equipment is of such a simple nature that much 
of it can be reproduced in the school itself, at little cost” (p. iv). LaZerte’s 
economical laboratory program was unlikely to be rejected because of its 
costly equipment, a consideration that did not seem to occur to Skinner and 


others who promoted expensive teaching machines and equally expensive 
instructional courseware. 
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Although prepared laboratory equipment packages were not offered 
from the publisher in later editions of Numbers Tell Their Story, the book 
remained in print until the mid-1960s (royalty statements in the LaZerte 
Papers, University of Alberta Archives). From this evidence, it appears that 
LaZerte’s innovation of laboratory mathematics was reasonably successful, 
although it was eventually eclipsed by devices and curricula developed by in- 
dividuals who took advantage of the mass-market distribution of publishers 
and manufacturers. By such means, many inappropriate and inferior de- 
vices and programs were, and still are, being foisted on educators as well as 
the general public. 


Implications 

In 1959, several years after his retirement, LaZerte wrote an article in 
which he foresaw little positive change in education and the repetition of 
previous mistakes because of a lack of innovation by educators. LaZerte 
(1959) believed that “too many of our colleagues are timid—afraid of 
change—afraid to use their professional freedom” (p. 49). LaZerte’s obser- 
vation is a likely explanation why various specialists in other disciplines 
have sought to improve education; it appears to them that little is being 
done within the discipline to improve it (Skinner, 1984). In addition, when 
it appears that the accomplishments of previous educators are being ignored 
or forgotten, others may fear that no progress has taken place. 

The development and marketing of devices to enhance learning has con- 
tinued during the last 50 years, with much impetus coming from the United 
States. While some of these devices have been useful in Canadian schools, 
others have been inappropriate because of their cost or incompatibility with 
the curriculum or school system (Rutherford, 1961). A review of the work of 
M.E. LaZerte indicates that as early as 1922, devices and materials to sup- 
port learning were being developed and researched in Canada. Although 
some of LaZerte’s research indicated that certain devices did not enhance 
learning in the most efficient manner, some of his devices and methods did, 
such as his Numbers Tell Their Story program, which was designed with 
Canadian schools in mind, particularl:; rural schools. LaZerte’s work indi- 
cates that Canadian educators are sufficiently competent and innovative to 
engage in such research and development; they do not necessarily have to 
follow a lead or trend originating in the United States. Furthermore, 
LaZerte’s experiments and findings suggest that there is a need for edu- 
cators to evaluate critically the effectiveness of such devices and materials 
in the context of the local classroom before they are widely accepted for use 
in the field of education. 
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The evaluation of student achievement is a characteristic element of formal classroom 
education. Educational measurement addresses issues of and develops procedures for stu- 
dent evaluation. Recently there has been an increase in expressed interest in the relation- 
ship between educational measurement and the classroom practices used in evaluating 
student achievement (Stiggins, 1988). Questions have been raised as to the utility of proce- 
dures developed through educational measurement to teaching and learning. There ap- 
pears to be a disparity between the science of measurement and the craft of evaluation as 
practiced in classrooms. 


This study attempted to shed some light on classroom evaluation practices by considering 
whether testing is a common classroom practice, what kinds of student abilities or charac- 
teristics should be measured, what methods are used to collect this information, and what 
is the extent to which external sources of instrumentation or data are of interest to 
teachers. The 1986 British Columbia Provincial Science Assessment (Bateson, Anderson, 
Dale, McConnell, & Rutherford, 1986) provided data that could address these issues. This 
study reports the data derived from the teacher questionnaires in regard to the student 
evaluation practices and attitudes of science teachers in British Columbia and discusses 
some implications for the improvement of applied educational measurement. 


The evaluation of student achievement is a characteristic element of formal 
classroom education. Tests, assignments, and projects are developed, ad- 
ministered, and marked by teachers in order to generate information about 
the achievement of individual students. Educational measurement is a field 
of development and research which addresses issues of and develops proce- 
dures for the evaluation of student achievement. These issues include test 
reliability, validity, standard error of measurement, item bias, test equating, 
and models underlying response data. A large number of formalized proce- 
dures have been developed for test design and construction, item analysis, 
test scoring and analysis, and standard setting. 

The relationship between educational measurement and classroom prac- 
tice should be one of utility: Educational measurement should produce in- 
formation and procedures which are useful in the evaluation of student 
achievement. Recently there has been an increase in expressed interest in 
the relationship between educational measurement and the classroom prac- 
tices used in evaluating student achievement (Stiggins, 1988). Questions 
have been raised as to the utility of procedures developed through educa- 
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tional measurement to teaching and learning. There appears to be a dis- 
parity between the science of measurement and the craft of evaluation as 
practiced in classrooms. 

Stiggins, Conklin, and Bridgeford (1986) noted in their extensive review 
of the literature on classroom assessment that the dominant paradigm in 
educational measurement is based on data generated from standardized 
paper-pencil tests used for the purposes of accountability. This focus of 
educational measurement is well illustrated in a recent description of the 
future of testing (Rudman, 1987). The article concentrates solely on pub- 
lished, standardized tests and does not mention the development and use of 
classroom tests by teachers. In contrast, Gullickson (1986) notes in his 
American study of teacher-perceived needs in measurement that teachers 
would favor more emphasis on the use of nontest methodologies in meas- 
urement and evaluation courses. These methodologies include rating scales, 
observation, and anecdotal records. Teacher interest in measurement is 
directed toward a variety of instruments and procedures developed for in- 
structional purposes and not toward the issues and procedures of main- 
stream educational measurement research and literature. 

As a specific example of the disparity between measurement and instruc- 
tion, consider a recent critique of the reporting of results of the National As- 
sessment for Educational Progress (NAEP) project in the United States 
(McLean & Goldstein, 1988). It was noted that, after a great deal of com- 
plex analysis involving large data sets collected over several years, NAEP 
was able to report reading results on a single scale. McLean and Goldstein 
point out that this modeling of reading ability as a single trait does not coin- 
cide with conceptions of reading abilities that are current among those in- 
volved in the teaching of reading. The measurement project (NAEP) pro- 
duces results that are meaningful within the measurement perspective of 
unidimensional human traits, but this perspective does not fit the multi- 
dimensional view of those involved in the study and teaching of reading. 

Although the focus of educational measurement is on large-scale testing 
projects and standardized tests, it appears that most of the achievement 
testing done is in classrooms by teachers as part of ongoing evaluation of 
student achievement. An estimate of the number of tests a student com- 
pletes during an academic career was provided by the results of a question- 
naire administered by the author in the fall of 1987 to a group of 92 fourth- 
and fifth-year University of Victoria students of an educational measure- 
ment course. The students were asked, “How many formal tests or ex- 
aminations (those counting for grades) would you estimate you have taken 
in your life?” The median response range was 500 to 600 such tests. The 
vast majority of these tests would not be standardized tests but rather tests 
developed by classroom teachers. In British Columbia, for example, a pro- 
vince that does have a provincial grade 12 examination program, a grade 12 
scholarship exam program, and a provincial learning assessment program, 
most students continuing through to postsecondary levels would write fewer 
than 20 such tests in their public school (K to 12) career. It is likely, then, 


124 


Evaluation of Student Achievement 


that most tests a student encounters are developed by classroom teachers to 
assess student achievement in specific instructional programs. 

The purpose of this study was to explore a facet of the relationship be- 
tween educational measurement and the evaluation of student achieve- 
ment. Because it appears that most of the testing to which students are 
exposed over their educational careers consists of procedures and instru- 
ments developed by classroom teachers to assess student performance in in- 
structional programs, a first step in exploring the utility of educational 
measurement is to investigate what goes on in the classroom. This could 
lead to the determination and perhaps the improvement of the fit between 
educational measurement and classroom practice. 

This study attempted to shed some light on classroom evaluation prac- 
tices by considering whether testing is a common classroom practice, what 
kinds of student abilities or characteristics should be measured, what 
methods are used to collect this information, and what is the extent to 
which external sources of instrumentation or data are of interest to 
teachers. The 1986 British Columbia Provincial Science Assessment (Bate- 
son, Anderson, Dale, McConnell, & Rutherford, 1986) provided data that 
could address these issues. The 1986 British Columbia Science Assessment 
investigated the current state of science education in the province through 
the administration of achievement measures and attitude scales to all stu- 
dents in grades 4, 7, and 10, and the administration of questionnaires to all 
teachers of grades 4, 7, and 10 science. Data collection took place in the 
spring of 1986. 

The teacher questionnaires focused on classroom practices and teacher 
attitudes toward various aspects of their teaching: teacher background and 
general information, coordination of science education, physical facilities, 
materials and equipment, science teaching, assessment and evaluation, and 
the science program. The grade 10 questionnaire contained 56 items. 
Grades 4 and 7 teachers were administered identical 34-item question- 
naires. 

This study reports the data derived from the teacher questionnaires in 
regard to the student evaluation practices and attitudes of science teachers 
in British Columbia. In total 1,895 grade 4 teachers, 1,519 grade 7 teachers, 
and 1,035 grade 10 teachers responded to the questionnaire. The grades 4 
and 7 teachers were essentially classroom teachers, teaching science to their 
homeroom class in addition to teaching a broad range of other subjects to 
these students. Grade 10 teachers tended to be specifically assigned to 
teaching science (Bateson et al., 1986). 

These data provide a start for a study of teachers’ practices in and at- 
titudes toward student evaluation. It is in this light that this article has 
been prepared. It is not to be inferred that these data fully answer the ques- 
tions posed. However, the responses to the limited number of questionnaire 
items by science teachers in British Columbia suggest directions the an- 
swers would take if a broader sample of teachers were asked a broader 
sample of questions and if a more comprehensive set of methodologies were 
employed. 
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Results 

In responding to the question: “Is testing a common classroom practice?” 
teachers of grades 4 and 7 science were asked to indicate how often 10 
classroom activities were carried out in a typical week. The results (Table 1) 
indicate that, of those activities listed, testing was the most frequent class- 
room activity, with approximately 70% of teachers reporting that students 
were tested “Always” or “Often” each week. Testing was a more frequent 
classroom activity than conducting science demonstrations or student ex- 
periments, using the computer, or doing homework. The pattern of respon- 
ses was similar for both grades 4 and 7 teachers. 

To answer the question of whether testing is a characteristic activity of 
the classroom, it is clear that testing is a common classroom activity. 

In considering what kinds of student abilities or characteristics should be 
measured, grade 10 teachers were asked about the extent to which each of 
the four science goal areas—Attitude, Skills and Processes, Knowledge, and 
Critical Thinking (Ministry of Education, 1981)—should be tested in the 
determination of student achievement. In general, teachers indicated that 
all four areas should be tapped for student evaluation purposes. 

Attitudes were considered to require less emphasis than the other areas, 
with over one fifth of the teachers indicating that Attitudes should not be 


Table 1 
Science Program — Activities in the Classroom 
Grades 4 and 7 


Item Percentage of Teachers 
Grade 4 Grade 7 


In a typical week of science classes, 


how often did you do the following? A OS N Av *Smeeni 
Carried out experiment for students 1 10 77 12 Oi bate Tae lS 
Handed out notes Isle mOZei35 0 AS36556 31 
Wrote out notes for students 20022) 60 6 223 E2013 
Students conducted experiments 
individually 2 Om (Ome tZ 222" Oe NG 
Students conducted experiments 
in small groups 2, 197 640015 RA) eS oe We 
Students used computer Usa eeleeco Ogee oie 
Students used library books 534255, 6/aages 2 1A 76: QO 
Students chose topics to study ZR S52 144 0° 6646; 748 
Students did homework 219 4 le Oo oon 


Students were tested 36 33 30 1 39 36 24 0 
1 
A= Always, O= Often, S= Sometimes, N= Never 
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Table 2 
Assessment and Testing — Curricular Goals Emphasis 
Grade 10 
Item Percentage of Teachers 


In the determination of the final grade, 
what emphasis should be placed on 
the following science goal areas? 


Attitudes 
0% 22 
10% 56 
20% 19 
30% 2 
40% 1 
50% 0 
60% 0 
Skills and Processes 
0% 1 
10% 10 
20% 31 
30% 44 
40% 12 
50% 3 
60% 0 
Knowledge 
0% 0 
10% 1 
20% 8 
30% 23 
40% 28 
50% wet 
60% 19 
Critical Thinking 
0% 1 
10% 22 
20% 46 
30% 23 
40% 6 
50% 1 
60% 1 


NN — —— 
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used at all in evaluating students. A majority of teachers (56%) indicated 
that Attitudes should count for 10% of the final grade. Essentially all 
teachers indicated that Skills and Processes, Knowledge, and Critical 
Thinking should receive emphasis in the determination of a science stu- 
dent’s final grade. The response modes were Skills and Processes, 30% em- 
phasis; Knowledge, 40% emphasis; and Critical Thinking, 20% emphasis. In 
summary, science teachers regard a number of different student skills and 
characteristics to be of importance for evaluation purposes. The weighting 
applied to the skills and characteristics varies from one category to another 
with Knowledge receiving the heaviest weighting and Attitudes the least. 

Science teachers were asked about the methods they used for evaluating 
student achievement. A number of common instruments and procedures 
were listed and the teachers were asked to indicate the emphasis they 
placed on each for student evaluation purposes. Table 3 lists the response 
frequencies for grades 4 and 7 teachers in relation to types of methods used 
to collect information about student achievement. Table 4 lists the response 
frequencies for grade 10 teachers. 


Table 3 
Assessment and Testing — Source of Assessment Data 
Grades 4 and 7 


Item Percentage of Teachers 
Grade 4 Grade 7 


What emphasis do you place on each 
of the below in deriving a final grade 


for a student in your science class? Se Se Neelata Saati. 
Anecdotal records A247 58° TH 16, 328846 6 
Teacher-made objective tests 1 3 48 48 Q, Pao 54 
Ministry of Education 
Classroom Achievement Tests G20 ee One 5S 225 elo. a 
Subjective tests 14a co Ome, 9 207° 50 10 
Activity/experiment write-up See) Om OeeeiiG 4, 41, Soave 
Individual work contracts Scenlame2oemer 54 15 24 8 
Reports on topics in science 6 15 68 11 G Ii G45 14 
Projects 4 9 70 17 4 14 64 19 
Oral tests 357.28); 92° 5 47> 3 lapel be 
Student self-reports Sanee |) 220 54) 21hwe) 2 
Attendance 6 13; 14) 6 67 Tieorn 6 


1 ' F 
N= No emphasis, L = Little emphasis, S=Some emphasis, M= Much emphasis 
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Table 4 
Assessment and Testing — Source of Assessment Data 
Grade 10 
Item Percentage of Teachers 


How much importance do you 
place on each of the following in 
deriving final student evaluation? Very Important Important Not Important 


Anecdotal records 


— achievement 19 39 41 

— general attitude 4 48 48 

— work habits 8 63 29 
Teacher-made objective tests 71 28 1 
Standardized objective tests 14 43 43 
Subjective tests 30 56 14 
Laboratory write-ups a7 59 3 
Individual work contracts 6 21 72 
Reports on topics 17, 64 20 
Projects 17 61 22 
Oral tests 4 26 70 
Student self-reports 3 23 75 
Attendance 17 50 33 
Classroom behavior 15 43 42 


Two trends emerge. First, teachers used a wide variety of methods to 
collect information related to student achievement: anecdotal records, sub- 
jective tests, experiment write-ups, reports, and projects. Second, in all 
grades, teacher-made objective tests received the most emphasis (and pre- 
sumably use) in the determination of student achievement. The Ministry of 
Education Classroom Achievement Tests, individual work contracts, oral 
tests, and student self-reports appear to be the least emphasized forms of 
assessing student achievement. The majority of grades 4 and 7 teachers 
(67%) did not use attendance for the determination of final grades. Yet 
most grade 10 teachers (67%) consider attendance to have at least some im- 
portance in the determination of a student’s final grade. 

In summary, teachers do not rely on a single source of data for evaluat- 
ing student achievement. They use a variety of methods to measure student 
performance with teacher-made objective tests receiving the most emphasis 
of those methods presented. 

Grade 10 teachers were asked about their preferences for using school, 
district, or provincial examinations for evaluation of student achievement. 
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The questions were asked in order to gauge teacher interest in and possible 
use of external sources of instrumentation. Teachers were asked the extent 
to which these various tests should “count” toward a student’s final grade 
(Table 5). There was considerable support for the use of a schoolwide exam. 
A majority of teachers (79%) thought that student assessment should in- 
clude the results of a schoolwide test counting for 20% or 30% of the final 
mark. However, 21% of teachers indicated that a schoolwide exam should 
not be used at all. In regard to the use of district or provincial exams, ap- 
proximately 75% of grade 10 science teachers indicated that they would not 
like to base student assessment on districtwide or provincial exams. It ap- 


Table 5 
Assessment and Testing — Instrumentation for Student Grade 
Grade 10 
Item Percentage of Teachers 
Would you like to have student 
assessment based on... 
schoolwide exam? 
No 21 
Yes, about 10% . 8 
Yes, about 20% 50 
Yes, about 30% 13 
Yes, about 40% 4 
Yes, about 50% 4 
districtwide exam? 
No 75 
Yes, about 10% 10 
Yes, about 20% 12 
Yes, about 30% 2 
Yes, about 40% 0 
Yes, about 50% 2 
provincial exam? 
No 77 


Yes, about 10% 
Yes, about 20% 
Yes, about 30% 
Yes, about 40% 
Yes, about 50% 


ao — o1 © OO 
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pears that teachers did not favor the use of either district or provincial level 
tests for student evaluation purposes. It should be noted that a teacher 
would likely have direct involvement with the development of a school 
exam, whereas the opportunity for direct involvement is somewhat less for 
a district exam and almost nil for a provincial exam. 

Teachers of grades 4 and 7 science were asked whether they had read 
any of the reports of the previous Provincial Science Assessment (Taylor, 
1982) in order to estimate their interest in external sources of data related 
to student achievement in science (Table 6). Reports had been produced at 
the provincial and district levels; no school or class level reports were 
produced in this provincial assessment. Most teachers (approximately 70% 
or more) indicated that they had not read any of the reports produced. 
These results suggest that external sources of data, in this case from a 
provincewide assessment of student performance in the subject they are 
teaching, are not of great interest or importance to teachers. 

In regard to the question of teacher use of external testing, the data 
presented here indicate that science teachers do not view positively the use 
of external testing for student evaluation. Nor do they have much interest 
in the data generated from large-scale assessments. 


Discussion 
The results suggest that testing is a common classroom activity and that 
there is substantial variation in the methods and formats used to estimate 
student achievement. Teacher-made objective tests are the most common 
form of instrumentation in use, although many other formats for data 
generation and collection are used by science teachers in British Columbia. 
Teachers believe that student achievement should be evaluated in a number 
of different domains: attitudes, knowledge, and skills. Further, teachers are 
not interested in the use of external forms of testing, either as instrumenta- 


Table 6 
Assessment and Testing — 
Utilization of 1982 Science Assessment Results 
Grades 4 and 7 


Item Percentage of Teachers Responding Yes 
Grade 4 Grade 7 


Have you read the following publications of 
the 1982 Provincial Science Assessment? 


District Interpretation Report 25 32 
Provincial Update flier 15 21 
Provincial Summary Report 25 32 


Provincial General Report 17, 22 
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tion or as the results of large-scale assessment, even in a subject they are 
currently teaching. 

The results pertaining to teacher test use are not surprising. In an ear- 
lier study of a broad sample of Ontario teachers (Wahlstrom & Danley, 
1976), it was found that teachers used a variety of methods in measuring 
student achievement and preferred to use instruments they developed 
themselves. However, Wahlstrom and Danley found that the preferred as- 
sessment technique of Ontario elementary teachers was observation of 
classroom work by the teacher, whereas secondary teachers prefer tests. 
These findings were similar to a recent American study (Stiggins & 
Bridgeford, 1985) which also found variation in preferences for student 
evaluation procedures by teachers across subject areas as well as across 
grade levels. 

The pervasiveness of testing in the classroom coupled with the use of a 
wide variety of formats to measure different kinds of student achievement 
does raise a number of important issues in regard to the relationship be- 
tween educational measurement and teacher practices in classroom assess- 
ment. To illustrate, consider one such issue: the combining of scores from 
different sources to arrive at a final grade. How do teachers combine the 
scores from a number of different testing formats when testing a number of 
different student abilities and characteristics collected over a period of 
time? Certainly the scores are combined, but how and with what effect? It is 
likely the task is conducted by simply summing up the numbers and assign- 
ing grades to these numbers. It would be of interest and utility to inves- 
tigate the procedures actually employed in order to determine their 
consequences and the consistency of implementation within and between 
classrooms. Nagy, Traub, and MacRury (1986) have described methods for 
research into the procedures used by teachers in determining lettergrades 
(standard setting). A description of these procedures and patterns of im- 
plementation would also illuminate some of the practices used by teachers, 
not only in the combining of scores to convert to grades, but also the at- 
titudes of teachers toward achievement, and an estimate of the variability 
of achievement standards. A number of such studies are in progress (Mac- 
Rury, 1988; Nagy, 1988; Traub, 1988; Wilson & Rees, in press). 

However, it is of concern that there is a widespread practice in student 
assessment that is not well addressed by educational measurement. It ap- 
pears that there is a substantial practice of combining test data from 
various sources, but the issue is largely ignored in the literature. On the 
basis of an informal review of educational measurement textbooks and jour- 
nals by the writer and also as noted by Stiggins et al. (1986), the literature 
focuses on single-test situations, that is, the development, administration, 
and analysis of a single test (or set of tests) as a discrete project. The most 
common manifestations of the single-test situation in schools are final ex- 
aminations, assessment programs (such as the one from which the data of 
this study were derived), and standardized test programs—areas that are of 
importance educationally and lend themselves to systematic research and 
development, but are of little interest or use to the classroom teacher (Table 
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6). Yet it is likely that for most teachers the measurement practices 
employed in the classroom are derived from personal experience both as 
teachers and as students, and from techniques garnered from colleagues. 
These classroom practices would form the basis of “the craft of evaluation” 
(McLean, 1985) and would not necessarily coincide with practices derived 
from the science of measurement and its literature. 

Assuming that most of the achievement testing being done in Canada is 
that done by teachers in classrooms, it appears that this practice could be 
better served by research. The apparent isolation of classroom testing prac- 
tice from research activity is a problem that should be addressed to improve 
overall educational practice. The determination and description of the 
measurement practices in current use and the extent of variation in prac- 
tice could lead to the development of better techniques for handling the data 
generated by classroom measurement practices and the generation of better 
data by classroom measurement. Because classroom derived technique 
predominates testing and student evaluation practices, applied educational 
measurement should center its attention on the classroom. 
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Although classical test theory has been a productive force in educational research for al- 
most a century, its impact on classroom practice in evaluation has not been especially 
strong. Little is known about what teachers actually do (as opposed to what they report 
doing). The policy and procedural context in which the evaluation of students occurs is 
similarly unexplored. The present study was conducted to examine what teachers do in the 
context of their classes and their school’s policies. Thirty-six teachers in a high school col- 
lected all their evaluation instruments for one class for a semester while maintaining a log 
and responding to a questionnaire for each instrument. Descriptive statistics for such 
categories as frequency of evaluation activity and purpose were computed. The results are 
discussed in terms of the complexity of the tasks these teachers faced in their evaluation 
practice. 


One of the few areas of research in education that has developed in a classi- 
cal, theory-driven manner is the evaluation of educational performance. 
Test theory, established over three quarters of a century, is presently alive 
on several fronts with empirical research. Examples include work in item 
response or latent trait (Hambleton & Swaminathan, 1985; Lord, 1980) and 
generalizability theory (Brennan, 1983; Cronbach, Gleser, Nanda, & 
Rajaratnam, 1972). 

Despite this activity, the relationship of research-based advances to 
school practice in student assessment has been and remains tenuous. Most 
evaluation texts and materials prepared for practicing educators adapt the 
test-building principles used by professionals (e.g., domain specification, test 
blueprinting, item format selection, piloting, item analysis, reliability, and 
validity checks) and assume that these adaptations, if made, would help 
teachers construct useful measures for a variety of classroom purposes 
(Brown, 1981; Cunningham, 1986; Thorndike, 1982). Yet one investigator 
(McLean, 1985), reviewing the “craft” of student evaluation in Canada, 
concluded: 


The elemental stuff of the measurement courses and textbooks is as foreign to class- 
room teachers as spectral analysis of glaze mixture is to potters—and apparently as 
irrelevant. (p. 39) 


One reason why this gap may exist is that test theory concerns itself 
primarily with issues of measurement (hence psychometrics). Because the 
purposes to which any instrument may be put are largely in the control of 
the users, the professional test producer can only anticipate the most com- 
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mon ones and provide as much relevant validity evidence as possible for 
those uses. As for the utility of the measure in promoting more effective 
educational decision making, the professional test producer is even more 
removed. Thus the conceptual model for professional test producers may be 
summarized as a fundamental concern with the technical characteristics of 
the measures (particularly as these affect the reliability of the scores); a 
large concern with validity in terms of anticipated usage (even though that 
usage is beyond the producer’s control); and a less-than-dominant concern 
for the overall utility of the measure in the various contexts in which it may 
be used. 

While all three of these aspects are likely to be present in school practice 
as well, their configuration will be quite different. Teacher-constructed in- 
struments are more likely to begin with decisions. Before teachers invest 
the time and resources necessary to develop a test, they will probably be 
cognizant of at least one specific need that the test is to serve. Because 
teachers are also more likely than test producers to know their clients, con- 
crete expectations about performance will affect the construction of the in- 
strument itself. (See Wilson & Rees, in press, for an elaboration of this 
model in more theoretical terms.) 

Consequently, a sensible order of events for a teacher-prepared test is 
likely to be the reverse of that of the professional, and the test preparation 
process will differ as well (Gullickson, 1986). The concern of the teacher will 
be centered on the decisions to be made from the instrument (the produc- 
tion of grades for reporting purposes and the documentation of work com- 
pleted are common examples) and relatively less on the characteristics of 
the measure itself. Much of this analysis is speculative, however, as little 
study beyond surveys of real behavior or the concepts which shape it has 
been conducted with teachers (Stiggins, Conklin, & Bridgeford, 1986). 

The research evidence that is available tends to partition itself into two 
categories. The first assumes that research findings could be most useful if 
they delineated what criteria and standards should be evident in classroom 
practice. In the inaugural issue of the Journal of Educational Measurement, 
Mayo (1964) documented the need for teacher understanding of such fun- 
damental measurement concepts as reliability and validity. How the needs 
were documented reflected a particular approach to improving teacher 
practice. Mayo identified instructors of measurement courses (as well as 
some others) and had them rate various statements concerning knowledge 
in the area in terms of perceived relevance to teachers. On this basis, 
presumably, courses and texts could be developed that would meet these 
needs. 

More recently, Rudman, Kelley, Wanous, Mehrens, Clark, and Porter 
(1980) continued to find large gaps in American teachers’ understanding of 
these concepts. Frisbie and Friedman (1987), accepting the gaps as given 
and as undesirable, have attempted to define the fundamental needs for 
classroom teachers in terms of an analysis of the Standards for Educational 
and Psychological Testing (AERA, APA, & NCME, 1985). Through the use 
of the standards, they argue, a domain description outlining the knowledge 
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structure fundamental to sound evaluation practice could be developed. 
Once developed, this structure could form the basis of teacher education in 
this important area. 

A second category of research, more recent in origin, attacks the issue 
from the perspective of the teacher. Researchers such as Gullickson (1984) 
argue that technical adequacy, for example, is only one of several factors in- 
fluencing teachers’ use of tests. Student acceptance and support of the test- 
ing model, the nature and quality of the information to be obtained from the 
test, external constraints, and the way in which the teacher intends to use 
test results may also be important factors influencing evaluation practice 
from the teacher’s point of view. 

In a series of studies by Stiggins and his colleagues (Stiggins & 
Bridgeford, 1985; Stiggins, Conklin, & Bridgeford, 1986), the issue of the 
applicability of the typical courses provided teachers in educational meas- 
urement has been raised. Teachers surveyed in these studies, for example, 
seem to rely most on the very measures (observation, assignments) that 
evaluation specialists relegate to minor status in their courses. The same 
teachers tend to view the results of standardized tests as only minimally 
useful, yet courses in measurement tend to stress the administration and 
interpretation of these measures far more than the preparation and inter- 
pretation of the techniques teachers use most often (Stiggins, Conklin, & 
Bridgeford, 1986). 

Although the issue of context is critical in the pursuit of this second ap- 
proach to research, few studies from this second perspective have collected 
the instruments teachers use. Thus far, most of the effort has been directed 
at results obtained from questionnaires. Secondly, there may be a need now 
to place the thoughts and actions of teachers into a more formal pattern. 
Perhaps teachers do have theories which guide their practice even if those 
theories are not fully developed, expressed, or matched to those of the 
evaluation professionals. 

The following investigation was pursued (a) in an effort to examine what 
some educators think and do in the context of their classes and (b) from this 
examination, to develop a model that might illuminate those actions. 


Method 

The investigation occurred in a high school situated in a relatively 
homogeneous, middle-class, suburban area of eastern Ontario. The pro- 
grams offered to approximately 600 students include “general” as well as 
“advanced” subjects, but the emphasis in the school is on academic courses. 
Typically, over 80% of the credits awarded will be of an advanced level. In 
any given year, 70% of this school’s graduates will enroll in postsecondary 
institutions. 

The evaluation study conducted in this high school resulted from a con- 
cern by a new principal that grades reported to parents be comparable from 
one subject to another. For example, the mathematics department, because 
of its commitment to a form of criterion-referencing, was typically produc- 
ing negatively-skewed distributions while others, particularly those in the 
humanities, were generating more symmetrical ones. A committee com- 
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posed of the principal, the author, and two teachers was charged with the 
responsibility of investigating what policies and procedures of evaluation 
would be most appropriate for the school. As part of this process, a survey of 
present practice was conducted. It is this survey that is reported on here. 


The Survey 


As part of the process of developing policies and procedures for the school, 
the committee decided to find out what evaluation practices were presently 
in force. Each of the 36 full-time teachers in the school collected every 
evaluation tool he or she used for one course for a full semester. 

All teachers participated in the data collection phase of the survey. Each 
was assigned one class for which the following data were required: an 
evaluation log documenting each instrument’s title, its date of use, the 
number of students who completed it, the numbers of As and Fs awarded, 
the median, total marks, and the number of classes participating; a copy of 
the instrument; and an evaluation checklist indicating the purposes of the 
instrument (from seven choices offered); an assessment of the achievement 
of the class in terms of teacher expectations; and any explanatory com- 
ments by the teacher concerning the context. 


Results 
Frequency of Evaluation 


The frequency with which students were evaluated was a primary concern 
of the school in terms of efficient use of instructional time and adequacy of 
assessment. (Table 1 indicates the total number of assessments per class as 
well as the frequency certain purposes were cited.) 

These data show that there was a large degree of variability from course 
to course in terms of the number of instruments used. However, for a typi- 
cal student averaging 107 hours of instruction per course in this semestered 
school, some evaluation would occur approximately every nine hours. For 
such a student carrying a full program of three to four courses, some type of 
formal evaluation activity was occurring approximately every second day of 
school. 

The frequency with which evaluation occurred could be related to other 
factors such as the subject matter and the level at which the courses were 
being taught. Table 2 displays the information concerning level and subject, 
with subject being dichotomized into “science and science related” and 
“humanities.” These two perspectives do not seem to account for the vari- 
ability observed in the data, thus indicating that it is the individual teacher 
whose view dictates the frequency of evaluation no matter what the course 
or level. 

This conclusion is reinforced by examining the correlations between fre- 
quency and purposes of evaluation displayed in Table 3. Those teachers who 
checked most often that their purposes for evaluating students included 
checking students’ progress against course objectives (Crit), generating 
marks for reporting purposes (Mark), diagnosing students’ weaknesses with 
material (Diag), and having students practice or apply what has been 
learned (App) were also those who tended to have more frequent evalua- 
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Table 1 
Frequency of Evaluation by Subject and Purpose During One Semester 
ee ee ce al ea a eee 
Teacher Purposes 
Crit Mark Diag App Cons Norm Next Total 
Science and 
Science Related 


Auto Mechanics 1 3 0 3 2 1 0 S 
Biology syohn) sete) m2 46 8 11 0 53 
Business oa 3 Ono mEeeG 2 0 3 
Business 11 10 8 9 7 ve 0 12 
Business 2 i 1 12 3 1 1 14 
Business 3 0 0 3 2 0 0 3 
Chemistry 4 4 2 4 4 1 0 4 
Computers 10 10 i, 8 5 0 0 10 
General Science 7 11 6 5 9 0 0 i8 
Mathematics 7 1 3 8 5 2 0 9 
Mathematics 9 15 13 12 9 6 1 cS 
Mathematics r¢ 8 8 1 0 9 
Mathematics 13 0 6 3 12 1 0 13 
Mathematics 12 14 12 0 0 0 lie 14 
Physics 13 14 6 18 17 6 6 22 
Physics 1 0 1 1 1 0 0 1 
Woodworking 2 2 1 2 1 1 0 2 


tions. Indeed, the high intercorrelations among these four categories indi- 
cate that they could be more adequately thought of as elements of a single 
general approach to evaluation. 


Objectives-Based Versus Norm-Referenced 


The orientation reflected in the tendency to give many evaluations often 
produces a negatively-skewed distribution of marks. The school policy, how- 
ever, required a more symmetrical view of what mark distributions should 
be. To ensure consistency of grading within the school, the following guide- 
lines had been issued by the principal to influence the reporting of grades: 


1. Course medians should normally be in the 66%-72% range at all grades 
and levels of difficulty. 


2. Failure rates should normally not exceed 15% in each course at all 
grades and levels of difficulty. 
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Table 1 (continued) 


Teacher Purposes 
Crit Mark Diag App Cons Norm Next Total 


Humanities and 


Social Sciences 

Art 8 0 2 fs) 8 0 0 9 
Drama 8 8 / 12 11 11 Z 12 
English 22) Onl0 11 188 15 1 0 25 
English 14 14 15 18 12 0 2 26 
English 18 21 16 21 22 4 11 23 
English O13 11 16 foe 7 if 6 19 
English 12 19 6 14 14 19 3 19 
English 5 11 5 8 7 0 0 13 
Family Studies 12 3 3 12 8 2 ¢) 15 
French 5 11 6 9 9 0) 0 12 
French 15 9 14 15 15 4 1 18 
Geography i 8 5 3 t 3 3 8 
History 8 9 6 4 8 3 0 10 
History 9 9 10 1 thee 10 3 0 11 
History 7 6 i 6 vi 2 0 a 
Music 10 10 10 10 10 10 4 10 
Phys. Ed. 1 1 1 2 1 1 0 2 
Phys. Ed. 10 10 0 6 0 0 0 10 
Phys. Ed. 1 1 0 1 1 0 0 1 


3. The number of first-class marks should normally equal or exceed the 
failure rate at all grades and levels of difficulty. 


This conflict in needs between administrative and instructional levels in the 
system was one of the main reasons for the total investigation, but of more 
importance to the present discussion was how the teachers were able to 
coordinate the contradictory purposes of “checking students’ progress 
against course objectives,” a goal more suitable to objectives-based assess- 
ment, with another goal of reporting grades where students’ achievement 
would be compared to others, a norm-referenced approach. 

One common solution was to use more frequent but less heavily 
weighted evaluations for the objectives-based purposes. These instruments 
typically had few failures (Fs) and many high marks (As). As the time for 
mark reporting drew near, more heavily weighted but less frequent tests 
and examinations were used to bring the total mark distribution into line. 
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Table 2 
Average Usage per Teacher by Subject and Level During One Semester 


N Overall By Purpose 
Crit Mark Diag App Cons Norm Next 


Science and 
Science Related 


Academic 12. 1391) (11730 10:38 9:0 99:2 6:5 2.5 STG 
General 3 ‘97 4 6.77% 3.0%s0 Bi.0 (Ss.0nmps 
Mixed 2 60 60 GO 4050 @3.0 wo.5 Meno 
Subtotal 17 116 98 918 7.4°%e5 I56 Go.amepe 
Humanities 
Academic 12 127 993 98" 68™ae —o2 “Male 
General 4 188 133 11.87 11.0%15.5 13.0 oommeare 
Mixed 3 7:7 57 3.0" 33) 7.7 = 6.7 eee eee 
Subtotal 19 132 95 91" 74100 =o6) mo cere 


Overall Total 86 124). 94> 9a 72m ome 3.1 1.4 


Note: Crit: criterion-referenced; Mark: mark generation; Diag: diagnostic; App: application; 
Cons: consolidation of knowledge; Norm: norm-referenced; Next: decide what to teach next 


These instruments usually had more of a balance between As and F's and 
medians within the range called for by the school’s policy or lower. (No 
doubt there are instructional reasons for this pattern as well: As more work 
is completed in a term, the capacity for more involved, integrative, and thus 
potentially more difficult, assignments increases.) 

The marks summary for one typical teacher is displayed in Table 4. Of 
the first six classroom-based assignments, totaling 80 marks, there were 
only five failures. The last six classroom-based assignments totaled 155 
marks and produced 13 failures. The final examination had only three As as 
well as three Fs, and a median at the lower end of the acceptable range of 
marks. Given the situation of a student receiving the median score on each 
of these 13 assignments, that student’s final mark would be 71%, an av- 
erage grade within the 65%-72% guideline but one in which most of the as- 
signments which had produced it had the negative-skewness characteristic 
of objectives-based measures. 


Discussion 
A most striking feature of the data concerning purpose is that evaluation in- 
struments invariably served at least two purposes, and frequently more 
than two. Part of this multiplicity of purpose is necessitated by the environ- 
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Table 3 
Means, Standard Deviations, and Correlations for Frequency and Purposes* 


Freq = Crit Mark Diag App Cons Norm Next 


Freq 12.44 
(9.65) 
Crit 93 9.42 
(8.96) 
Mark .89 .87 9.11 
(9.32) 
Diag .89 .93 .92 7.22 
(8.97) 
App .93 .87 .87 .87 9.28 
(8.33) 
Cons 57 42 is 38 (56 peel 
(5.41) 
Norm .45 42 .53 42 .53 .46 3.03 
(4.23) 
Next 25 15 .29 .20 a5 .38 .46 1.47 


(2.95) 


Note: Crit: criterion-referenced; Mark: mark generation; Diag: diagnostic; App: application; 
Cons: consolidation of knowledge; Norm: norm-referenced; Next: decide what to teach next. 
Correlations in bold are significant beyond the .01 level. 

*Means and Standard Deviations on diagonal 


ment set by school policy. Instruments under this system cannot be con- 
sidered as discrete evaluations; they must fit into an emerging pattern 
resulting in conclusions of overall performance called grades, conclusions 
that involve normative judgment. Thus the professional testmaker’s con- 
cern with clarity and singleness of purpose, a unitary index of reliability, 
and a view of validity based on the administration of a single instrument 
may not have much applicability in the complexity represented by this 
milieu. 

While both professional testmaker and teacher may share a concern for 
reliability in measurement, validity in judgment, and utility of decision 
making, the teachers in this school at least were also manipulating addition- 
al pressures related to focus. The students, the institution, and the teacher 
as self-evaluator all brought different demands to the exercise of evaluation. 


Focus on Students 

The need to make decisions about student progress (both formatively and 
summatively) precipitates much of the evaluation activity within the school. 
This dominant concern is reflected in the purposes checked, many of which 
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Table 4 
Summary of One Teacher’s Marks Record 


Assignment Number Number Median Total 
Number of As of Fs (in %) Marks 
1 if 1 73 15 
2 4 0 15 10 
3 14 0 92 20 
4 0 3 70 10 
g 4 1 15 10 
6 18 0 96 15 
ii 5 74 72 40 
8 8 5 62 30 
9 8 1 70 10 
10 10 1 100 10 
11 2 2 70 15 
Wa 5 2 64 50 
13 3 3 65 100 


could be subsumed under the general category of monitoring achievement, 
of providing objectives-referenced assignments and norm-referenced tests 
that generate marks leading toward a final, summary assessment. 


Focus on Institution 


From the school’s point of view, this summarizing decision making should 
occur in a consistent form across subject areas, grades, and levels of pro- 
gram. Thus the school shapes the structure of its reporting system through 
its policy statements concerning median ranges, failure rates, and sym- 
metricality of score distributions. 

The acceptance of this school policy influences the focus teachers are 
able to maintain with their students. Knowing they must provide individual 
assessments, day to day as well as end of semester, teachers create expecta- 
tions for their students that, among other things, contribute to the grading 
structure set up for the school as a whole. As Table 4 illustrates for one 
teacher, assignments tended to be easier, less heavily weighted early in the 
semester leading to a greater reliance on summative, more heavily weighted 
evaluations as the end of the course approached. The mandating of a formal 
examination period at the end of the semester for all courses also en- 
couraged this pattern. 


Focus on Teacher 


Finally, the data indicate that the teacher may also use the results to review 
the course itself. This attribute may explain why 58% of the instruments 
were checked by teachers as having as a purpose “to diagnose students’ 
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weaknesses with material” (Diag) but only 12% of the instruments were 
seen as useful “to decide what to teach next (Next).” 

The teachers may have mentally recorded items needed for review 
without using that information formally with the class or classes which 
generated the data for that particular conclusion. Such a review would like- 
ly shape the teacher’s own standards of expected, subsequent performance 
both for himself or herself and the students. The instruments and their re- 
sults, then, may have given some teachers feedback on their own perfor- 
mance, leading them to conclusions about what to do differently in the 
future. 


Conclusion 
While the data collected in this study are closer to the best evidence 
criterion than those typically collected through objective means, it is still 
unclear whether teachers view activities of evaluation in a coherent, holistic 
manner. 

At present, further investigations are being conducted which elicit ra- 
tional and contextual data from more than one site and from repre- 
sentatives of more than one level within the system. It is only in such a 
comparative situation, with data solicited both qualitatively and quantita- 
tively, that firmer conclusions concerning the impact of levels, for example, 
could validly be seen to be a differential and powerful influence on the 
evaluation of student achievement. 

Nevertheless, the present analysis does provide some putative responses 
to frequently posed but seldom answered questions such as why teachers 
seem reluctant to adopt ready-made instruments for classroom use and why 
they so seldom apply the models of evaluation represented in measurement 
texts. The ready-made instruments may perform one part of the’ teacher’s 
purpose for evaluation but not another. Such tests might, for example, pro- 
vide for some relevant comparative data but not fit into the pattern re- 
quired by the teacher for grade reporting. Similarly, the professional’s con- 
cern with reliability of scoring, item analysis, and systematic content 
coverage may also seem of only esoteric interest, and even an imposition, to 
teachers practicing the delicate balancing acts required by the different, 
sometimes contrary, forces operating on them already. 
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A six-month study of writing performance contrasted pencil-and-paper (P&P) with lab- 
based word processing (W-P) for regular grade (5 to 8) children (N=36). In terms of 
productivity, W-P increased (a) number of drafts, story units, and overall output for all 
students; and (6) final draft length for grades 5 and 6 students. In terms of revisions, W-P 
increased (a) spelling, paragraphing, and formatting revisions; (b) meaning-preserving 
revisions for grade 6 students; and (c) meaning-altering, deletion revisions for grade 8 stu- 
dents. Strong linear trend effects were evident indicating developmental improvement. 
Deviation from linearity was due to notably poorer performance for the lower grade stu- 
dents, particularly grade 5 students. Grade 5 students appear significantly less able to use 
W-P technology. Overall, the results present an argument for the use of W-P at the elemen- 
tary and junior high school levels and indicate that a lab setting is workable. 


The purpose of this study was to explore word processing effects on writing 
in normal-achieving children. The rationale for this focus emerged from (a) 
the call for answers to questions on word processing effects as a precursor 
to the tendering and testing of theoretical hypotheses (Logan & Padro, 
n.d.), (b) the call for research to supplement the sparse data which are 
presently available on word processing effects (Collier, 1983; Grabe, 1985; 
Maddux, 1985), and (c) the call for clarification of the effects of word pro- 
cessing on revision (Fitzgerald, 1987). 
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The availability of a word processor does not presage positive effects. Re- 
search on composing is less than encouraging. Programs that interact with 
the writer by asking for input to generate a poem may teach poetic struc- 
ture (Owens, 1984) yet may detract from creativity and originality. 
Moreover, programs that prompt the writer by asking directive questions in 
response to “help” keys or time delays may actually retard the composing 
process (Woodruff, Bereiter, & Scardamalia, 1981-82). Furthermore, there 
is some evidence that children may not be predisposed to compose on a word 
processor; rather, they may prefer to compose with pencil and paper and 
revise on the word processor (Woodruff, Lindsay, Bryson, & Joram, 1986). 

A second problem concerns the measurement of word processing effects. 
Qualitative effects are often evaluated merely by amount written or subjec- 
tive judgments (Bradley, 1982; Kleiman & Humphrey, 1982). In addition, 
the reported research effects are often more descriptive than statistical 
(MacArthur & Shneiderman, 1986; Sapona, Lloyd, & Wissick, 1986). In ef- 
fect, qualitative improvement is not always evident (Woodruff et al., 1981- 
82) or is not clearly evident (MacArthur & Shneiderman, 1986; Sapona et 
al., 1986). Thus it is essential to have measures which convincingly dem- 
onstrate quantitative and qualitative effects. 

The third problem concerns target populations. Existing word processing 
research is not child oriented (Bridwell, Nancarrow, & Ross, 1984; Collier, 
1983; Fitzgerald, 1987). 

Thus there seem to be legitimate concerns about the existing research 
base (Collier, 1983; Grabe, 1985; Maddux, 1985). There is a need to address 
these concerns when exploring word processing effects with children. The 
present study uses a comprehensive within-subjects design with a thorough 
statistical analysis in an attempt to transcend the limitations and subjective 
views in word processing research. 

Intuitively, word processing seems to have the potential to enhance the 
writing of both normal achieving and communication disordered children. 
Beyond such benefits as motivation and aesthetically appealing printed 
products, word processing allows all students adult-like writing experiences 
(Papert, 1980). It has been suggested that word processing may facilitate 
language development (Bradley, 1982) and cognitive development (Alex- 
ander, 1983). Moreover, the central craft components (Graves, 1983) of 
writing may be improved by word processing technology. 

In fact, word processing may prove to be one of the most significant 
educational applications of the microcomputer. It eliminates typewriters, 
pens, scratch paper, erasers, cut-and-paste tools, and possibly even dic- 
tionaries, moreover, the simplification of storing, editing, and revising text 
leaves students free for higher order processing (Kleiman & Humphrey, 
1982). 

Higher order aspects of creative writing include planning, composing, 
and editing-revising. It is the simplification due to the editing-revision 
process which appears to have the potential to facilitate creative writing by 
allowing for a constructive focus on content and revision (Graves, 1983). 
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The tedium associated with error corrections, text modifications (changes, 
additions, deletions, etc.), and subsequent drafts is eliminated. 

Furthermore, there would be a recursivity established between thinking 
and writing by means of the revision process (Alexander, 1983). The fact 
that the writer cyclically moves back and forth between thinking and writ- 
ing, while revising, may benefit both cognitive and language processes. 

Given the existing research deficiencies and the intuitive appeal of word 
processing, a study was designed to explore word processing effects with 
children. Developmental data were collected on the writing performance of 
normal achieving children (grades 5-8) in both pencil-and-paper and word 
processing settings. The focus of the study was limited to the editing- 
revision process (in view of the findings of Woodruff et al., 1986) rather 
than the planning or composing processes. Consistent with the logic of 
popular opinion on the potential value of word processing, it was predicted 
that (a) word processing would increase productivity in terms of drafts 
produced, story units produced, number of words written on final drafts, 
total word output and revisions made, and (b) proficiency would increase 
with grade level. 


Method 
Subjects 


Ten subjects were randomly selected from each of four grades (5-8) and 
matched by teachers according to sex, age, and achievement to form two 
groups at each grade level. Two subjects at each of the grades 5 and 6 levels 
were lost due to attrition and incomplete data, leaving a sample of 36 stu- 
dents. The teachers used the Whole Language approach for the language 
arts program. Writing activities emerged from integrated thematic studies 
where topics could be suggested by the teacher or selected by the student. 
Even so, the writing program was not considered to be a significant variable 
because a within-subjects design was used with appropriate counterbalanc- 
ing for order of word processing exposure. 

Test scores on the Test of Written Language (TOWL, Hammill & Lar- 
sen, 1983) indicated that student writing proficiency was within the av- 
erage range (85 to 115, standard score mean=104.5). 


Procedure 

Each student was required to use the microcomputer for writing for a 
period of three months (approximately 30 sessions at 30 to 40 minutes per 
session) and, further, to use paper-and-pencil only in the writing program 
for a comparable three month period involving the same number of 30 to 40 
minute sessions. One group of five students at each grade level was ran- 
domly selected to do the paper-and-pencil sessions first, while the second 
group did the word processing sessions first. Groups were reversed at the 
end of the three-month period. Because the purpose of the study was to ex- 
amine the revision process, first drafts of all creative writing were done in 
pencil-and-paper sessions under teacher direction. Subsequent drafts were 
done in either pencil-and-paper or word processing, depending on group as- 
signment. 
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Five microcomputers (Commodore 64s) were set up in an empty class- 
room along with one 1526 dot matrix printer. One additional computer with 
a Panasonic printer was set up in another room for final printing. Students 
(grades 5 to 8) in groups of five were booked into the computer lab for 30- 
minute sessions daily. Teachers monitored student writing from their class- 
rooms and only occasionally had the opportunity to monitor the lab. Lab 
assistance was provided by a cooperative education student on work-assign- 
ment from a secondary school or by a knowledgeable grade 8 student. All 
students were entering, saving, and printing text within their first two ses- 
sions using the menu driven word processor Better Working Word Processor 
(Spinnaker, 1985). 

A standardized writing test (TOWL) was used for pretest and posttest 
comparisons. On subtests used from this test, students were required to (a) 
complete sentences using a cloze format which evaluated for Word Usage 
(the use of correct verb tense, auxiliary verbs, number, comparatives, etc.); 
(b) proofread and correct sentences for punctuation and capitalization 
which evaluates for style; and (c) to write a story in response to a pictorial 
stimulus which was evaluated for Vocabulary (words used with a length 
greater than six letters) and Thematic Maturity (based on 20 questions 
relating to the use of humor, personal names, dialogue, etc.). Story length 
was also tabulated and recorded. Technical data on the TOWL for stand- 
ardization, reliability, and validity are reported to be satisfactory (Salvia & 
Ysseldyke, 1981). All reliability coefficients reported in the manual exceed 
.80. Moreover, where a subjective element is involved (Thematic Maturity) 
interscorer reliability is reported as .93. With respect to validity, the au- 
thors report adequate criterion-related and construct validity. 

The classroom teachers—who were unaware of the specific objectives of 
the study—monitored the writing of all students and provided motivation 
and editorial feedback. However, the feedback during word processing was 
less immediate in view of the teacher-student distance factor. Students in 
the lab were required to be more independent. The students revised their 
text in the lab and presented their revised draft (i.e., a printout) to the 
teacher for further editorial comment. The process continued until both 
teacher and student were satisfied with the draft which then became the 
final draft. The teachers coded each draft as first, second, third, and so on 
and filed a copy for subsequent evaluation for developmental changes in 
quantity and quality. 

To determine productivity (i.e., quantitative changes) in both pencil-and- 
paper and word processing settings, frequency counts were made for (a) 
number of drafts produced (i.e., number of revised printouts on a story 
which were presented to the teacher for evaluation and feedback); (b) num- 
ber of words in each draft; (c) number of story units (i.e., “final publica- 
tions”) generated over the three-month period; and (d) total output of each 
group in each setting. 

To evaluate revisions (i.e., qualitative changes) a taxonomy modeled 
after Faigley and Witte (1984) was developed (Table 1). Text was evaluated 
for Infralexical Changes and Lexical Changes. The Infralexical Changes 
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Table 1 


Taxonomy of Revision Types 
See a eee fever oleh: one min WE bere Seine 


Infralexical Revisions Lexical Revisions 
ee ee A TO 200 DONS ORO cae OP IO Ay IoA | 
Level of Change Level of Change 
A. First-Order Changes A. First-Order Changes 
(meaning-preserving) 
1. Spelling (01) 1. Additions (11) 
2. Punctuation (04) 2. Deletions (12) 
3. Substitutions (13) 
B. Second-Order Changes B. Second-Order Changes 
(meaning-altering-gist-preserving) 
1. Tense, Number, etc. (02) 1. Additions (21) 
2. Abbreviations (03) 2. Deletions (22) 
3. Substitutions (23) 
C. Third-Order Changes C. Third-Order Changes 
1. Paragraphing (05) 1. Additions (31) 
2. Formatting (06) 2. Deletions (32) 


3. Substitutions (33) 


Note: The numbers in brackets are cross-references to the taxonomy used by Faigley and 
Witte (1984). 


were considered to be below the level of the “word” and were catalogued as 
First-Order Changes (spelling and punctuation), Second-Order Changes 
(tense/number and abbreviations), and Third-Order Changes (paragraphing 
and formatting). 

The Lexical Changes were considered to be at the “word” level or above 
and involved additions, deletions, or substitutions. At the First-Order level 
the changes were considered to be meaning-preserving; at the Second-Order 
level the changes were defined as meaning-altering but gist-preserving; and 
at the Third-Order level the changes were considered to be meaning-alter- 
ing and gist-altering. By way of explanation, a gist-alteration in a passage 
would result from a change in the body of the text (e.g., addition) that would 
also require a change in the written “abstract.” A change in the body of the 
text which changed the meaning but did not require a change in the 
“abstract” would be a gist-preserving change—a Second-Order change. 

To apply this taxonomy in a manageable way and within a standardized 
format, the first 30 words in each revised draft were evaluated for revisions 
and then tabulated. The rationale for a 30-word limit emanated from the 
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need to keep the data manageable and the observation that there were no 
written story units smaller than 30 words in length. 

Pretest and posttest scores using standardized scores (i.e., standard 
scores) from four subtests (Word Usage, Style, Vocabulary, and Thematic 
Maturity) of the TOWL were evaluated for both pencil-and-paper and word 
processing conditions. 


Results 
The data were analyzed using the mixed-model analysis of variance 
(ANOVA) with repeated measures, followed by trend analysis (F-tests) and 
simple effects tests (t-tests) where appropriate, The results are presented in 
four sections: Writing Productivity, Lexical Revisions, Infralexical Revi- 
sions, and Test of Written Language. - 


Writing Productivity 

Number of drafts. A two-way (4x2) ANOVA with Grade (5, 6, 7, 8) and 
Setting (word processing, pencil-and-paper) as the independent variables 
was computed for number of drafts produced. Setting was a within-subjects 
variable treated as a repeated measure. There was a main effect for Setting, 
F(1,32)=30,40, p<.0001, with the word processing grouping producing 
more drafts (mean=2.54) than the paper-and-pencil grouping (mean= 
1.78). Also, there was a main effect for Grade, F(3,32)=40.13, p<.0001, 
with no interaction effects. As may be seen in Figure la, however, there was 
a strong linear trend, F(1,32)=54.24, p<.01, with an increase in drafts at 
the higher grade levels. Deviation from linearity, F(2,32)=33.0, p<.01, was 
apparently due to lower performance overall at the lower grade levels and a 
plateau at the higher grades for pencil-and-paper. Simple effects tests con- 
firmed (a) that the grades 5 and 6 students were performing lower than the 
grades 7 and 8 students, p<.01; and (b) only the grade 5 students did not 
show a significant difference between word processing and paper-and-pencil 
settings, p>.05. 

Number of story units, A two-way (4x2) ANOVA with Grade and Setting 
as the independent variables was computed for number of story units 
produced. There was a main effect for Setting, F(1,32)=5.68, p<.05, with 
the word processing grouping producing more story units (mean=11.92) 
than the paper-and-pencil grouping (mean=10.1), and a main effect for 
Grade, F(3,32)=4.98, p<.01. The significant linear trend for Grade, 
F(1,32)=6.62, p<.05, was affected by significant deviation, F(2,32)=4.16, 
p<.05. Tests for simple effects revealed lower performance in story unit 
production for grades 5 and 6 students (Figure 1b) when compared to the 
grades 7 and 8 students but only during the paper-and-pencil sessions, 
p<.05. Thus word processing seems to increase story unit production for 
lower grade students. 

Total words produced. For total output a two-way (4x2) ANOVA (Grade 
by Setting) was computed for number of words produced. There was a main 
effect for Setting, F(1,32)=24.28, p<.0001, with the word processing group- 
ing producing more words (mean=4560.75) than the paper-and-pencil 
grouping (mean=2711.33). Also, there was a main effect for Grade, 
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Word Processing 
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(b) MH Word Processing 
Pencil-and-paper 
Mean 
Number 
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(c) @B Word Processing 
Pencil-and-paper 
6500 
Total 
Gime 500 
umber 4c 
3500 
Words 2500 
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500 ; 33 
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(d) 8 Word Processing 
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Figure 1: Written productivity for students in grades 5 through 8 during word 


processing and paper-and-pencil sessions in terms of (a) number of drafts produced, 


(b) number of story units generated, (c) total number of words produced, and (d) 


number of words written for first and final drafts 
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F(3,32)=18.44, p<.0001. The significant linear trend for Grade, F(1,32)= 
26.27, p<.01, was affected by significant deviation, F(2,32)=14.51, p<.01. 
In the paper-and-pencil sessions, simple effects tests revealed lower perfor- 
mance by grades 5 and 6 students when compared to the grades 7 and 8 stu- 
dents, p<.01. In the word processing sessions all students appeared to 
benefit (Figure 1c); however, the difference for grade 5 students between 
pencil-and-paper and word processing was not significant, p>.05. 

Draft lengths. A three-way (4x2x2) ANOVA with Grade, Setting, and 
Draft as the independent variables was computed for length of first and 
final drafts. There was a main effect for Grade, F(3,32)=4.39, p<.01, which 
was qualified by a two-way interaction for Grade and Draft, F(3,32)=3.47, 
p<.05, and a two-way interaction for Setting and Draft, F(1,32)=7.21, 
p=.01. As may be seen in Figure 1d, the grades 7 and 8 students showed in- 
creases in final draft length, whereas the grades 5 and 6 students showed 
decreases. Tests for simple effects revealed that the grade 8 students 
showed final draft increases in both word processing, p<.04, and pencil- 
and-paper settings, p<.05, whereas the grade 7 students showed increases 
in the word processing sessions only, p<.03. Furthermore, the grade 5 stu- 
dents showed a dramatic absence of final draft length during pencil-and- 
paper sessions, p<.005, but not during word processing, p>.05. A linear 
trend was not evident for overall draft length, F(1,32)= 2.1, p>.05. Means 
and standard deviations for number of drafts, number of story units, num- 
ber of words written, and length of first and final drafts are reported in 
Table 2. 


Lexical Revisions 


A four-way (4x2x3x3) ANOVA with Grade (5, 6, 7, 8), Setting (word 
processing, pencil-and-paper), Level of Change (First-Order, Second-Order, 
Third-Order), and Revision Type (addition, deletion, substitution) as the in- 
dependent variables was computed. The final three variables were within- 
subjects variables treated as repeated measures. There were main effects 
for Grade, F(3,32)=34.96, p<.0001; Level of Change, F(2,64)=42.14, 
p<.0001; and Revision Type, F(2,64)=20.64, p<.0001; and two-way inter- 
actions for Grade and Setting, F(3,32)=3.87, p<.02; Grade and Level of 
Change, F(6,64)=10.37, p<.0001; and Grade and Revision Type, F(2,64)= 
5.95, p<.0001. In addition, there were three-way interactions for Grade, 
Setting, and Revision Type, F(6,64)=3.83, p<.003; and Grade, Level of 
Change, and Revision Type, F(12,128)=2.08, p<.02. To clarify these inter- 
actions, each Level of Change was analyzed separately. The descriptive data 
on these separate effects are presented graphically in Figure 2. 

First-order lexical revisions. A 4x2x3 ANOVA (Grade by Setting by 
Revision Type) on the First-Order revisions revealed main effects for Grade, 
F(3,32)=13.55, p<.0001; and Revision Type, F(2,64)=5.95, p<.005; and 
two-way interactions for Grade and Setting, F(3,32)=5.78, p<.003; and 
Grade and Revision Type, F(6,64)= 3.17, p<.01. The linear trend showing 
increased revisions at the higher grade levels was significant, F(1,32)=6.37, 
p<.05, as was the deviation from linearity, F(2,32)= 17.14, p<.01. Tests for 
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Table 2 
Means and Standard Deviations for Word Processing 
and Pencil-and-Paper Sessions at Four Grade Levels* 


Grade 5 Grade 6 Grade 7 Grade 8 

Number of Drafts 

W-P 1.8 (1.0) 1.9 (0.3) 3.0 (0.5) 2 (0.6 

P&P 1.1 (0.3) 1.3 (0.4) 2.2 (0.3) 2.2 (0.4 
Number of Stories 

W-P 9.1 (4.7) 13.6 (4.1) 12.4 (2.0) 12.3 (3.6) 

P&P 7.0 (3.6) 7.8 (4.3) 12.0 (2.8) 12.7 (3.9) 
Number of Words 

W-P 1623 (1039) 4082 (1381) 5580 (1964) 6275 (2528) 

P&P 833 (616) 1718 (1485) 3708 (1385) 4012 (1699) 
First Draft Length 

W-P 107.4 (38.0) 163.5 (86.7) 143.1 (57.8) 148.0 (47.7) 

P&P 101.5 (26.6)  218.9(279.5) 139.1 (43.0) 131.6 (50.0) 
Final Draft Length 

W-P 94.8 (47.3) 154.7(43.5)  1683(66.0) 173.8 (66.8) 

P&P 9.4 (26.6) 63.3 (69.6) 149.1 (52.8) 162.3 (85.5) 


Note: W-P = Word Processing and P&P = Pencil-and-Paper. 
*Standard Deviations in brackets 


simple effects revealed no differences in the revision incidence for grades 6, 
7, and 8 in the word processing setting; however, the grade 5 students 
showed a lower incidence of revision on all three revision types, p<.05. In 
the pencil-and-paper sessions there were no differences between the grades 
5 and 6 students nor between the grades 7 and 8 students. However, the 
grades 5 and 6 students showed a lower incidence of revision than the 
grades 7 and 8 students, p<.01. Thus it was the grade 6 students who 
seemed to be more able to take advantage of the word processing setting 
when compared with their pencil-and-paper performance. 

Second-order lexical revisions. A 4x2x3 ANOVA (Grade by Setting by 
Revision Type) on the Second-Order revisions revealed main effects for 
Grade, F(3,32)=37.08, p<.0001; and Revision Type, F(2,64)=13.11, 
p<.0001; a two-way interaction for Grade and Revision Type, F(6,64)= 
3.47, p<.01; and a three-way interaction for Grade, Setting, and Revision 
Type, F(6,64)=3.57, p<.005. Linear trend was not significant (F'<1). Tests 
for simple effects revealed lower revision incidence for grades 5 and 6 stu- 
dents when compared to grades 7 and 8 students in the word processing set- 
ting, p<.05, and the pencil-and-paper setting, p<.01 (Figure 2b). When 
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BME Word Processing 
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tc) Ml Word Processing 
See Pencil-and-paper 
1.75 
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Figure 2: Mean number of lexical revisions (additions, deletions, and substitutions) 
at the First-Order level (11, 12, 13), the Second-Order level (21, 22, 23), and the 
Third-Order level (31, 32, 33) for students in grades 5 through 8 in word processing 
and pencil-and-paper sessions in the first 30 words of revised drafts 
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grade level was considered separately, only grade 8 students showed a dif- 
ference between word processing and pencil-and-paper, and there only for 
deletion revisions which increased during the word processing sessions, 
p<.01. 

Third-order lexical revisions. A 4x2x38 ANOVA (Grade by Setting by 
Revision Type) on the Third-Order revisions revealed main effects for 
Grade, F(3,32)=13.50, p<.0001, and Revision Type, F(2,64)=7.94, p<.001, 
which were qualified by a two-way interaction for Grade and Revision Type, 
F(6,64)=4.30, p<.01. Linear trend was significant F(1,32)=4.21, p<.05, as 
was deviation from linearity, F(2,32)=18.14, p<.01. In terms of grade level 
the grades 7 and 8 students showed superior revision incidence for addition 
revisions only, p<.01, in both the word processing and pencil-and-paper set- 
tings (Figure 2c). Means and standard deviations are reported in Table 3. 


Infralexical Revisions 


At the Infralexical level, a four-way (4x2x3x2) ANOVA (Grade by Setting by 
Level of Change by Revision Type) was computed. There were main effects 
for Grade, F(3,32) =11.61, p<.0001; Setting, F(3,32)=21.93, p<.0001; 
Level of Change, F(2,64)=50.07, p<.0001; and Revision Type, F(1,32)= 
1.21, p<.005. Two-way interactions were evident for Grade and Level of 
Change, F(6,64)=3.11, p<.01; Setting and Level of Change, F(2,64)=8.58, 
p<.001; and Setting and Revision Type, F(1,32)=6.26, p<.02. Also, there 
was a three-way interaction for Setting, Level of Change, and Revision 
Type, F(2,64)=12.50, p<.0001, and a four-way interaction for Grade, Set- 
ting, Level of Change, and Revision Type, F(6,64)=2.87, p<.02. To clarify 
these effects, each Level of Change was analyzed separately. The descriptive 
data on these separate effects are presented graphically in Figure 3. 

First-order infralexical revisions. A 4x2x2 ANOVA (Grade by Setting by 
Revision Type) on the First-Order revisions (spelling, punctuation) revealed 
main effects for Grade, F(3, 32)=5.79, p<.003; Setting, F(1,32)=11.39, 
p<.0083; and Revision Type, F(1,32)=6.70, p<.02. There was a two-way in- 
teraction for Setting and Revision Type, F(1,32)=15.75, p<.001, and a 
three-way interaction for Grade, Setting, and Revision Type, F(3, 32)=3.11, 
p<.05. Linear trend was significant, F(1,32)=4.30, p<.05, as was the devia- 
tion from linearity, F(2,32)=6.52, p<.01. Simple effects tests on the 
revision type revealed that the higher revision incidence in word processing 
was due to the increase in spelling revisions, p<.001 (Figure 3a), although 
this was not true for grade 5 students, p>.05. Again, the grade 5 students 
seemed less able to effectively use the word processor beyond the paper-and- 
pencil level. 

Second-order infralexical revisions. A similar three-way (4x2x2) ANOVA 
(Grade by Setting by Revision Type) on the Second-Order revisions 
(tense/number, abbreviation) revealed main effects for Grade, F(3,32)= 
8.32, p<.001; and Revision Type, F(1,32)=25.15, p<.0001. Also, there was 
a two-way interaction for Grade and Revision Type, F'(3,32)=7.30, p<.001. 
Linear trend for grade level was significant, F(1,32)=5.54, p<.05, as was 
the deviation from linearity, F(2,32)=9.70, p<.01. Simple effects tests on 
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Table 3 
Lexical Revisions with Word Processing and Pencil-and-Paper 


_ itu ae Sa eg Oe ee 
Word Processing Pencil-and-Paper 
Grade Grade 


First Order 
Additions M 025 4.75. 1.10 ‘2.00 0.035.023) 6140 aoa 
sD 0.46 +.4:75... 4.204 A49nmslts 35a 1e20 e2.a7 
Deletions M 0.4 3:0 15d 2100 elt STON GiLAOn SAENSOw 1.20 
SD 0.85. 1.18-21.15> 1.87 0.305 20-35 o 7a 
Substitutions M O43, 2.63. .2.26 .1.20,..0.38) 0:00.73 00382250 
SD 0.385i02:72> 4:83 1.48 © 1,06) 70,000) 2:007 1858 
Second Order 
Additions M 0.139:,1.13. 3.90..4:20 .0.00. 0 2535022550 
sD 0,35+-41146203:519%2: 705, 0.00) (046i B.58) 2:80 
Deletions M 0713" .0:13. “1220 F250" 0:00R208 S 128080. 30 
SD 0:35 0:35> 1.40 70°88" 0:00" 0.65 asl 4000110 
Substitutions M 0.00. 0.75..210 1/40. 0313) G00 3-00) 3:50 
SD 0.00 AAG, 1.3 F405 1)%, 0:35-0:00 see 452.84 
Third Order 
Additions 0.00 0.13 1.30 0.80 0.00 0.00 2.00 1.40 
0.00 0.35 1.06 0.92 0.00 0.00 1.83 1.90 
Deletions 0.00 0.00 0.10 0.70 0.00 0.00 0.20 0.40 


Substitutions 
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the revision types revealed a higher incidence of tense/number revisions, 
p<.01; however, this was due to the grades 7 and 8 students, p<.05, not the 
grades 5 and 6 students (Figure 3b). Because setting was not significant, 
word processing does not seem to affect Second-Order Infralexical revisions 
at the grade levels studied. 

Third-order infralexical revisions. A 4x2x2 ANOVA (Grade by Setting 
by Revision Type) on the Third-Order revisions (paragraphing and format- 
ting) revealed main effects for Grade, F(3,32)=3.47, p<.03, and Setting, 
F(1,32)=44.00, p<.0001. As may be seen in Figure 3c, word processing 
clearly facilitates Third-Order revisions. There were no interactions. Linear 
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Figure 3; Mean number of infralexical revisions for students in grades 5 through 8 
in word processing and pencil-and-paper sessions in the first 30 words of revised 
drafts. First-Order level: spelling (01), punctuation (04); Second-Order level: tenses 
(02), abbreviations (03); Third-Order level: paragraphing (05), formatting (06) 
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trend for grade level was not significant. Simple effects tests on the revision 
types revealed that the lower incidence of revision in word processing for 
grade 5 students differed significantly only from the grade 8 students, 
p<.05. However, in view of the general equivalence in performance between 
grades 6, 7, and 8 students (Figure 3c), the main effect for Grade would ap- 
pear to be due to the poor grade 5 performance. Means and standard devia- 
tions are presented in Table 4. 


Test of Written Language 

Standard scores. A four-way (4x2x2x4) ANOVA on the TOWL standard 
scores (Table 5) with Grade (5, 6, 7, 8), Setting (word processing, pencil- 
and-paper), Test Time (pretest, posttest), and Subtest (Vocabulary, The- 
matic Maturity, Word Usage, Style) as the independent variables, with the 
last three variables being within-subjects variables treated as repeated 
measures, revealed no significant effects for Setting. A main effect for Test 
Time, F(1,32)=15.54, p<.001, indicated higher posttest scores in both set- 
tings. A main effect for Subtest, F(3,96)=7.21, p<.001, reflects generally 
lowered performance on the Style Subtest relative to the norming group. 


Table 4 
Infralexical Revisions with Word Processing and Pencil-and-Paper 


Word Processing Pencil-and-Paper 
Grade Grade 


First Order 
Spelling 


M 
SD 

Punctuation M 1.25 2.00 4.20 4.80 0.50 1.63 2.70 4.40 
SD 


Second Order 

Tense, etc. M 0.00 0.00; 1.10) 1.50 0.00 0:25 94.10) 1.30 
SD 0:00) 0:00%. 1:37 1.78. 0.00) 0:74 Ai 10. ihe 

Abbreviations M 0:00 0.00: 0:10: 0:00; 0:00: 0:00 6:207°040 
sD 0.00 0.00 0.32 0.00 0.00 0.00 0.42 0.32 

Third Order 

Paragraphing 0.50 2.00 1.40 1.30 0.00 0.00 0.25 0.70 


M 
SD 

Formatting M 0.38 1.75 1.60 2.50 0.00 0.00 0.00 0.50 
SD 
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Table 5 
Pretest and Posttest Standard Scores on the Test of Written Language 
(TOWL) for Word Processing and Pencil-and-Paper Conditions 


Word Processing Pencil-and-Paper 
Grade Grade 
5 6 is 8 5 6 7 8 


Pretest 
Vocabulary M isa O25 10.811 Sid tal 108 Oe 14 
sD SOR 351) bg, (29-42 O20" Se ees 
Thematic M POG e156 2010/2" 10°82 10:61 0:9 49:97 10:4 
Maturity SD Coe tls 2. ek eT DAD ee Le 
Word Usage M 1 a OA a My ey Ya hs See ec Pam Dec PR | We premen BOe:| 
SD ZA8y SattG a2 34 Cie HbtyediBa ocSio 125 
Style M 95 105 66 97 9.6" 106 784° [02 
sD 09 1.1 S08 2 1 ee elite To 0 wee 
Posttest 
Vocabulary M 1Oior1 0.82 107 2a 20 2516-6 ° CS Peete 14 
SD 3.0 3.1 Fs i ade te ls a al ac Shey dag yy 
Thematic M OOleiiso 10a 10-9 10.0. Te. 10 ed ad 
Maturity SD Ole eek. | ie? a 2. Oe 2-8 ea ee Poe 3 ee ek 
Word Usage M DsBetS.6 Fy 11902 14.41 he aS tise 12 
sD See SSE BY 2 OTM 287 Sait (6 Settee 2 fF 
Style M 2) 3 Ts 0) eg SS ae © J giana * © Desay By PS Heer.’ Batwa: 
SD 8 Onda 2B us? Gomes 1a Tells Ogee 


Note: The standard scores have a mean of 10 and a standard deviation of 3. 


There were no interactions. Thus revision mode does not influence written 
language test performance. 

TOWL story lengths. A three-way (4x2x2) ANOVA (Grade by Setting by 
Test Time) for the number of words written on the TOWL pretests and 
posttests was nonsignificant, thus indicating all groups were writing stories 
of comparable length both before and after the word processing and pencil- 
and-paper sessions. 


Discussion 
The prediction of increased productivity with a word processor was sup- 
ported. When students were assigned to the word processing session they 
produced (a) more draft copies of revised texts, (b) more story units, and (c) 
more words in total output. Furthermore, the word processing setting 
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facilitated the production of final drafts, especially for grade 5 students. 
This is consistent with previous research indicating increased final draft 
length following revision (Morton, in press; Woodruff et al., 1986). Thus 
word processing is seen to be of value in producing quantitative changes in 
writing. 

The qualitative changes were more localized. On Lexical Revisions, ef- 
fects were noted which seemed to indicate developmental factors. First, 
word processing facilitated the meaning-preserving revisions for the grade 6 
students in particular. As may be seen in Figure 2a, grades 5 and 6 students 
showed an absence of meaning-preserving (First-Order) revisions during 
pencil-and-paper sessions. However, during word processing sessions the 
grade 6 students showed an increased revision rate comparable to grades 7 
and 8 students. Apparently, the grade 6 students have the ability to use 
meaning-preserving revisions, but the paper-and-pencil setting is an inter- 
fering rather than a facilitating factor. The grade 5 students appear to lack 
this proficiency regardless of mode of output. Conversely, the grades 7 and 8 
students manifest proficiency in either mode of output. Thus the grade 6 
students show particular qualitative benefits from the use of the word 
processor. 

Second, the grade 8 students showed an increase in meaning-changing 
deletion revisions during the word processing sessions. This may be espe- 
cially significant in that text deletion is a low frequency revision charac- 
teristic. Graves (1983) claims that, “it is a long time before any writer 
spontaneously wants to delete information” (p. 159). He further points out 
that, developmentally, text deletion occurs with teacher help prior to inde- 
pendent deletion practices. Yet in the present study, text deletion was seen 
to be contingent on the word processing setting. Thus this sophisticated 
aspect of word processing may reflect a developmental maturity factor for 
grade 8 students which is allowed to surface independent of teacher direc- 
tion because of the word processing technology. 

On Infralexical Revisions, word processing led to (a) an increase of spell- 
ing revisions, although this was not true for grade 5 students; and (b) a 
general increase of both paragraphing and formatting revisions. Thus, at 
both the Infralexical level and the Lexical levels, word processing is seen to 
benefit writing. 

The prediction of developmental progression was also supported. In- 
creasing linear trend over grade levels was generally evident. The deviation 
from linearity appeared to be due to several clear grade level differences. 

Generally the grades 5 and 6 students were less proficient than the 
grades 7 and 8 students. Moreover, intragrade analysis showed that grade 5 
students were not able to use the word processor to exceed paper-and-pencil 
performance levels. This may be an artifact of curricular differences, or it 
may reflect a discrete developmental difference in acquiring and using word 
processing skills. Grade 5 students in particular may be too young to take 
advantage of lab-based word processing. 
inion es at times the lower grade level students responded favorably to 

processing sessions (e.g., by increasing the final draft length). This 
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indicates a potentially positive effect for word processing. Similarly, the 
grade 6 students generated more story units during word processing. How- 
ever, it is not clear whether this is due to motivation or to the word process- 
ing technology. 

With respect to revisions, developmental effects were evident also. 
Generally the grades 5 and 6 students showed reduced revision proficiency. 
The grade 5 students were notably lacking on infralexical revisions—spell- 
ing, paragraphing, and formatting. Again, this would seem to suggest that 
the grade 5 students are more limited in effectively using the word proces- 
sor in a lab setting. 

Overall, however, word processing appears to facilitate the productivity 
and revisional aspects of writing. This finding adds to the existing research 
on revision (Fitzgerald, 1987) by showing not only that word processing has 
positive effects, but also that these positive effects extend, in some ways, to 
young children. Older students may be using the technology more effective- 
ly, but all students were seen to benefit from the word processing setting. 
Introducing systematic word processing practices to elementary school set- 
tings, therefore, would seem to herald positive effects. 

However, the current lack of computers presents a problem (Maddux, 
1985; Zaharias, 1983). Maddux (1985) notes that elementary students av- 
erage about 30 minutes per week of computer time. He suggests that 
“schools may be adopting a policy of giving very light exposure to a large 
number of students, rather than intensive exposure to a smaller number” 
(p. 29). Such a time distribution would not be conducive to word processing. 
However, in the present study an intensive exposure was provided success- 
fully to small groups of students. 

The computer lab may be an ideal way to intensify exposure. By slotting 
time in a lab, the entire school may be able to increase access to computers. 
The process would be more systematized and computer “down time” would 
be minimal. Simple arithmetic suggests that 20-minute time slots in a lab 
with five computers could accommodate 70 students per day. If students 
had access to a time slot every day, or every other day, small schools could 
probably meet the needs of upper grade level (grades 6, 7, and 8) students. 
Larger schools would need larger or multiple labs. 

Ideally, a teacher-run lab would alleviate monitoring problems and allow 
for effective interaction and conferencing. In fact, Morocco and Neuman 
(1986) see the main advantage of word processing in the access the teacher 
has to the process the child is using in writing. The text is more public, thus 
allowing for a more interactive process with other students and the teacher. 

Unfortunately, the expense of a teacher-run lab makes it unlikely at the 
elementary level. However, computer labs could be attached to a library set- 
ting (with a computer-literate librarian) or to remedial and special educa- 
tion settings. Also, peer monitors, cooperative education students, and par- 
ent volunteers may work well in a lab setting as mediators between teach- 
ers and students. 

The effects in this study are encouraging and serve to justify further ex- 
plorations and innovations that would support word processing at the 
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elementary school level. For example, future research could examine the in- 
terface between specific writing programs and word processing technology. 
Some approaches to writing might be more amenable to the use of a word 
processor. Another possibility relates to the viability of using a lab setting 
with communication disordered students. Word processing technology may 
be a valuable tool in helping learning disordered students who have difficul- 
ty with writing. Also, a more stringent analysis of the cognitive processes 
involved during word processing would be an appropriate pursuit. This 
could be accomplished on the one hand by introducing “think-aloud” 
protocols and on the other hand by shifting the focus to include the plan- 
ning and composing stages of writing on a word processor. In fact, a great 
deal of research remains to be undertaken in the field of computer applica- 
tions (Logan & Padro, n.d.). 
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Levels of Performance and 
Cognitive Interference in 
Test-Anxious Subjects 


The major purpose of this study was to determine whether highly text-anxious subjects, in 
situations that pose testlike challenges, perform at relatively low levels and experience 
relatively high levels of task-irrelevant thoughts. The Test Anxiety Scale (TAS) was used 
to distinguish high and low test-anxious subjects. The subjects were individually ad- 
ministered the Wisconsin Card Sorting Test (WCST) as a measure of cognitive perfor- 
mance. Immediately after the WCST, subjects were asked to complete the Cognitive 
Interference Questionnaire (CIQ) which assesses irrelevant thinking that interferes with 
concentration on a task. Significant differences were found between the cognitive perfor- 
mance of high and low test-anxious groups, with high-anxious subjects performing less 
adequately. The high test-anxious subjects also experienced significantly more cognitive 
interference or task-irrelevant thinking than did their less anxious counterparts. A sig- 
nificant positive relationship between level of test-anxiety and amount of cognitive inter- 
ference experienced was established. 


According to the theoretical position adopted for this research, worry and 
other task-irrelevant cognitive activities associated with anxiety impair the 
quality of task-relevant performance. One explanation for the deterioration 
in performance is that the task-irrelevant information involved in worry 
and cognitive self-concern competes with task-relevant information for 
space in the processing system. As a consequence of critical self-focusing, 
highly test-anxious subjects are in effect in a dual-task or divided attention 
situation, in contrast to nontest-anxious subjects who primarily process 
task-relevant information. 

This study compares levels of negative, self-deprecatory thoughts in high 
and low anxious groups and further explores the impact of this task-ir- 
relevant interference on performance. Unlike many test-anxiety studies 
that have measured performance in terms of overall success or failure, this 
study examines particular components of learning that may inhibit or fac- 
ilitate overall task performance. Inefficient initial conceptualization, per- 
severation, conceptual-level responses, failure to maintain set, and 
inefficient learning across several stages of a test are all evaluated as pos- 
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sible factors involved in the anticipated performance decrements of highly 
test-anxious persons. 

Deffenbacher (1978) investigated sources of interference in highly test- 
anxious subjects who were solving difficult anagrams. Major findings were 
that, under evaluative stress, the high-anxiety group: (a) reported more 
anxiety during testing; (b) rated themselves, their abilities, and the task 
more negatively; (c) solved fewer anagrams; (d) experienced more inter- 
ference from anxiety; and (e) reported greater distraction of attention due 
to heightened autonomic arousal (emotionality), worrisome thoughts 
(worry), and task-produced competing responses (task-generated inter- 
ference) than did low-anxiety individuals under evaluative stress. 

In a review of the literature on paper-and-pencil anxiety scales (Sarason, 
1960), a number of studies were cited that provide evidence that highly test- 
anxious subjects are more “deprecatory, more self-preoccupied, and general- 
ly less content with themselves than subjects lower in the distribution of 
test anxiety scales” (p. 404). In addition to studies based on paper and pen- 
cil measures of anxiety, other studies indicate the self-deprecatory, self- 
ruminative tendencies of highly test-anxious subjects. In one such study 
(Sarason, 1960), highly test-anxious and low anxious college students were 
required to describe themselves for approximately a half an hour. Nonrein- 
forcement, reinforcement of negative self-references, and reinforcement of 
positive self-references were compared. An interesting finding of this study 
is that although highly test-anxious subjects are extremely responsive to the 
reinforcement of negative self-references, they show no response to the 
reinforcement of positive self-references. 

A large-scale investigation by Many and Many (1975) found a significant 
negative relationship between self-reported self-esteem and test-anxiety. 
Several studies provide even more direct support for the contention that 
high test-anxiety is associated with self-preoccupations and self-devaluing 
cognitions during task performance. Mandler and Watson (1966), Neale 
and Katahu (1968), and Marlett and Watson (1968) administered several 
tasks to high and low test-anxious individuals. In each investigation, 
posttask questionnaires asked subjects to indicate the incidence of self- 
devaluing cognitions during task performance. In all three studies, to the 
question, “How often during the testing did you find yourself thinking how 
well, or how badly you were doing?” the high test-anxious subjects reported 
a significantly higher incidence of such thoughts than did the low test- 
anxious persons. 

Wine (1980) found that reassuring students improves their performance 
and that even relatively trivial manipulations, such as suggesting to stu- 
dents that they concentrate more on a task and less on extraneous matters, 
improves performance. Consistent with the framework promoted by this re- 
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searcher, such instructions enable students to devote a greater proportion of 
processing capacity to the task and less to anxious self-ruminations. The en- 
hancement of performance may be attributable to having more cognitive 
capacity available for task solution. 

The most direct evidence for the high incidence of interfering self- 
relevant thoughts during test-anxious persons’ task performance was 
reported by Ganzer (1968). He recorded the task-irrelevant comments of 
high and low test-anxious individuals during a serial verbal learning task. 
The highly test-anxious subjects made many more irrelevant comments 
than did the low, and the comments were mostly of an apologetic self- 
deprecatory nature. 

It is clear that not only is test anxiety associated with a general tendency 
to be self-deprecatory, but that negative self-devaluing cognitions are 
elicited in test-anxious individuals during evaluatively stressful cognitive 
task performance. It is suggested that the cognitive representation of these 
negative self-preoccupations must absorb some portion of the person’s in- 
formation processing capacity, thus leaving less capacity for coping with 
task demands. In this sense, anxious individuals must divide their attention 
between task requirements and irrelevant cognitive activities such as worry 
and self-criticism. With only partial attention devoted to the cognitive task, 
decrements in performance are anticipated, particularly on complex tasks 
that require complete attention for success. In this study the cognitive per- 
formance adequacy of high and low test-anxious subjects was measured and 
discussed in relation to levels of negative task-irrelevant thoughts. 


Method 
Subjects 


The subjects were 214 students enrolled in an introductory psychology or 
sociology course at Grant MacEwan Community College. Three separate 
college campuses were involved in this study, with samples of students from 
south, central, and west Edmonton, Alberta. Grant MacEwan Community 
College offers one- and two-year programs in areas such as social work, law 
enforcement, and general arts for adults in postsecondary education. Twen- 
ty-one (nine males and 12 females) and 18 (nine males and nine females) 
volunteers were operationally defined as high and low test-anxious respec- 
tively on the basis of the Test Anxiety Scale (TAS). The subjects ranged in 
age from 18 to 46 years with a mean age of 27 years. 


Procedure 


The TAS (Sarason, 1973) was administered to the students, and the upper 
(scores>26) and lower (scores<9) 30% of the scale distribution operational- 
ly defined high and low test-anxiety groups respectively. The selection of 
score ranges for grouping subjects was influenced by several factors. First, 
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the score criteria and cut-off points that were used to define groups ac- 
counted for the anticipated attrition of subjects. Second, the mean anxieties 
of these groups are comparable to those of other studies by Sarason (1978) 
and Deffenbacher (1978). Finally, homogeneity in amount of test-anxiety 
experienced was established within low and high test-anxious groups (stan- 
dard deviations of 2.3 and 3.1 respectively). 

The 39 students, of which 21 were defined as high test-anxious and 18 
defined as low test-anxious, were individually administered the Wisconsin 
Card Sorting Test (WCST) one to two weeks after the TAS was admin- 
istered. It was initially explained to each participant that the author was 
conducting a research study designed to investigate individual reactions to 
tests. Students were told that the WCST was a measure of one’s ability to 
do college-level work, and participants were instructed to perform to the 
best of their ability, as they would be compared with others. The WCST was 
presented in this way in an attempt to simulate an achievement-oriented 
situation. 

The WCST provides six scores related to cognitive processing: inefficient 
initial conceptualization, number of perseverative responses, percent per- 
severative errors, percent conceptual level responses, failure to maintain 


Table 1 
Means and Standard Deviations for High and Low 
Test-Anxious Groups on WCST Subscales 


Anxiety Level WCST Subscales 
1 2 3 4 5 6 


High Anxious X 21.56 16.67 11.33 rays 0.78 = =t33 
Shei741 9.11 1.71 149 0.81 9.00 


Low Anxious X 32.62 27.19 35.71 6.29 2.14 -3.05 
Ses 2.1. 10.91 40.85 2.59 1.56 9.44 


Effect Size a ifs3 =1.05 =1519 0.68 1.14 0.19 


WCST Subscales 

1 =Number of perseverative responses 
2= Percent perseverative errors 

3 = Inefficient Initial conceptualization 
4 = Percent conceptual-level responses 
5= Failure to maintain set 

6 = Learning across stages of the test 
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set, and inefficient learning across the stages of the test. The format of the 
test requires the subject to sort a pack of 64 cards on which are printed one 
to four symbols (star, cross, circle, or triangle), in green, red, yellow, or blue, 
under four stimulus cards (one red triangle, two green stars, three yellow 
crosses, and four blue circles) according to a principle that the subject must 
deduce from the pattern of the examiner’s responses to the subject’s place- 
ment of cards. For example, if the principle is color, the correct placement 
of a blue card is under four blue circles, regardless of the symbol or number. 
The examiner responds only with a “right” or “wrong” statement. After the 
subject has placed 10 consecutive cards according to the required principle, 
the examiner changes the principle being used for placement, indicating the 
change only in the changed pattern of the right and wrong statements given 
to the subject. The test continues until the subject has made six runs of 10 
correct placements, has placed more than 64 cards in one category, or has 
used 128 cards. | 

Immediately after the WCST, the Cognitive Interference Questionnaire 
was administered. The CIQ (Sarason & Stoops, 1978) was designed to as- 
sess the degree to which people, after working on a task, report having had 
thoughts that interfere with concentration on the task. The CIQ consists of 
22 items; the first 21 are rated for the frequency of occurrence of particular 
types of intrusive thoughts. The first 10 items pertain to the task on which 
the subject has just performed, and the remaining items refer to a diversity 
of thoughts unrelated to the task. The final item provides a global rating on 
a 7-point scale of the degree of mind-wandering experienced while working 
on a task. Testing time varied from 20 to 40 minutes per participant. 

Two questions were asked in this study: (a) Does any difference exist be- 
tween high and low test-anxious subjects’ performance on the WCST? (b) 
Does a difference exist between the high and low anxiety groups as reported 
on the CIQ? 


Results 
Anxiety and Cognitive Performance 


The Hotelling’s T* was used to test the difference between groups on the six 
WCST scores, and the observed value of 25.46 was significant (p<.007). 
Overall, the test of the vector of means of the latent variable—cognitive 
processing—revealed significant differences between low test-anxious and 
high test-anxious groups. However, post-hoc tests failed to highlight any 
single variable as significant. Thus the six variables of the WCST as a group 
generate significant differences between high and low test-anxious groups, 
but no one variable is strong enough to explain the differences. 
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Level of Anxiety and Cognitive Interference 


The TAS and the CIQ were used to measure level of test anxiety and 
amount of cognitive interference respectively. A comparison between the 
mean cognitive interference scores for the two anxiety groups indicated that 
high anxious individuals experienced a greater amount of cognitive inter- 
ference than did their low anxious counterparts (t=5.66, df=37, p<.0001). 


Discussion 

The results of this study confirmed the anticipated performance decrements 
of highly test-anxious subjects but failed to identify a specific variable that 
was responsible. Also, as predicted, the high test-anxious group experienced 
greater interference from anxiety than did the low test-anxious group. 
Thus, for the highly test-anxious, an evaluative situation appeared to elicit 
interfering anxiety in the form of attention to worrisome thoughts and 
ruminations and focus on elements of the task irrelevant to efficient prob- 
lem solution. The low test-anxious group, however, did not experience this 
dysfunctional pattern of attention. 

The results of this study suggest a particular approach to the treatment 
of test-anxious persons, the goal of which is to reduce the negative self- 
focus, thus enabling students to devote a greater proportion of processing 
capacity to the task. The specific concern with how highly test-anxious sub- 
jects use their task time suggests treatment in which subjects are not only 
given intensive practice in dealing with tests, but also that this task practice 
should be accompanied by instructions to focus fully on the task and to in- 
hibit self-relevant thinking, There is a large body of literature in selective 
attention, the basic premise of which is that the subjects’ attention can be 
differentially directed to specific stimuli or away from others. With repeated 
training under attention-direction instructions, subjects become more and 


Table 2 
Cognitive Interference: Means and Standard Deviations 
for Low Anxious and Hiah Anxious Groups 


Anxiety Level X SD t 
High Anxious 50.381 12.785 

5.66% 
Low Anxious 31.889 7.194 


*p <.0001 
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more skilled in attending to the “relevant” stimuli and ignoring the “ir- 
relevant” stimuli (Wine, 1971). Further research is required to determine 
the degree to which attentional training not only improves task perfor- 
mance but lowers reported test anxiety levels as well. 

Cognitive therapies implemented by school counselors or teachers offer 
effective coping strategies for test-anxious students. One such strategy in- 
volves replacing negative self-talk with positive self-talk, for example, “I am 
prepared; I have gone over this material thoroughly and will give this test 
my best shot.” Cognitive therapy can also be used to counteract exaggerated 
irrational thoughts regarding possible negative test outcomes. 

A weakness in the area of test anxiety research is that there has been no 
careful analysis of the nature of performance decrements experienced by 
test-anxious subjects. Focus has been placed merely on overall success or 
failure while ignoring particular sources of difficulty. The WCST, which 
provides scores for six variables, was used in this study in an attempt to 
overcome this weakness. Future researchers need to recognize and define 
the properties of specific tasks and the attentional demands they make on 
subjects. 
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Experience: Satisfaction and 
Success of Students Who Transfer 


College transfer students attending the University of Alberta were surveyed to determine 
which variables were most closely associated with student success and satisfaction in their 
transfer from college to university. Both interviews and questionnaires were used. Stu- 
dents tended to be more satisfied with the college experience than with the university ex- 
perience. Most students reported using a different form of support system at university 
than at college. Students who were more successful at university had a greater preference 
for a transfer program than did the less successful students. Implications for practice are 
related to availability of program counseling and transfer information. Suggestions for 
further research include the development of a student decision making model. 


In varying degrees, universities in North America have attracted students 
who begin their studies in a college setting and, after a period of study, 
move to a degree-granting university to complete their programs. These 
students and their programs are a significant presence in contemporary 
postsecondary education. The purpose of the present study was to examine 
some of the factors that influence the satisfaction and success experienced 
by students who attend college before university. 

For several decades, transfer education has received wide attention in 
the literature. Dennison (1986) stated that Canadian college enrollments 
have grown continuously in recent decades and noted that the transfer 
function of British Columbia colleges has been challenged by the addition of 
other college programs. He observed, however, that although colleges offer 
increasingly diverse programs to larger numbers of students, the number of 
British Columbia college students annually registered in transfer programs 
during this period has remained in the 7,500 to 9,000 range. 

On the American scene, Cohen (1985, p. 152) found that college enroll- 
ments have grown continuously during the past four decades. For this peri- 
od, Knoell (1982) concluded that the transfer function of American colleges 
has been diminished by the expansion of other college programs. The 
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change in college program offerings appears to reflect changes in the types 
of expectations that have been placed on colleges. One consequence of the 
greater demand being placed on the college system in Canada and the 
United States may be that transfer students are not receiving as much in- 
stitutional support as they once did. 

Research has suggested that transfer students may experience special 
problems in completing their educational programs. Dennison and Jones 
(1970) identified lack of academic preparedness, loss of course credits, and 
lack of prerequisites as problems affecting British Columbia students in the 
1960s. Fetters (1977), in a national longitudinal study in the United States, 
found that two-year transfer-in students withdrew more frequently from a 
four-year campus than did students who began their study on a four-year 
campus. Cross (1968), Cohen and Brawer (1981), and Rich (1979) con- 
cluded that ethnicity, finances, residence location, and admissions policies 
also contributed to the particular problems experienced by transfer stu- 
dents. 

The purpose of this article is to report a recent study of transfer students 
attending the University of Alberta and to discuss implications of findings 
about transfer student satisfaction and success for the postsecondary sys- 
tem in Alberta. 


Relevant Literature 

Studies of the transfer function have progressed over time from initially in- 
vestigating issues involving transfer of credits (Cross, 1968; Knoell & 
Medsker, 1965) to measuring completion rates in baccalaureate degree pro- 
grams (Dennison, 1974) to present research interests of identifying 
problems experienced by transfer students in the transfer process (Remley 
& Stripling, 1983). Literature related to university transfer students has 
been summarized in the areas of (a) transfer in Alberta and (b) satisfaction 
and success in university. 


Transfer in Alberta 


In a critical analysis of the Canadian college system, Dennison (1986) ob- 
served that there is no national college-level system of education for 
Canada. Rather, he noted that the role for colleges differed in each province, 
with the transfer function strongest in Alberta, British Columbia, and 
Quebec. 

In September 1974, at the request of the Universities Coordinating 
Council and presidents of institutions of higher education in Alberta, the 
Minister of Advanced Education announced the establishment of a Council 
on Admissions and Transfer (Alberta Council on Admissions and Transfer, 
1981). From the several pressing transfer-related issues, the Council de- 
cided that college-university transfer problems were a first priority. This 
priority was addressed by preparing and publishing a Provincial Transfer 
Guide for the 1976-1977 academic year. The guide, now entitled Alberta 
Transfer Guide, is published annually to reflect the current status of 
negotiated arrangements between colleges and universities in the province 
(Alberta Council on Admissions and Transfer, 1987). 
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A transfer patterns study conducted in 1983 provides the most recent 
figures about the volume, direction, and nature of postsecondary student 
transfer in Alberta. The findings of this study (Alberta Advanced Education, 
1984) revealed that the largest proportion of transfers over a five-year 
period had been to universities. Of the 5,136 students who transferred in 
the postsecondary system in 1979, 4,144 students transferred to univer- 
sities. By 1983, out of a systemwide total of 7,078 student transfers, the 
number of university transfers had grown to 6,987, an increase of 69% com- 
pared to a systemwide increase of 38%. According to the Council, “the 
single institution receiving the highest number of full-time transfers each 
year was the University of Alberta” (Alberta Advanced Education, 1984, p. 
22). 

After a three-year study of students transferring from an Edmonton 
community college to the University of Alberta, Small and Konrad (1986) 
concluded that there were sufficient issues about student persistence and 
satisfaction to warrant further study. 


Satisfaction and Success in University 


The study of student satisfaction and performance is related to, but distinct 
from, worker satisfaction and performance. Tinto (1986) succinctly de- 
scribed this difference and cautioned against direct application of job satis- 
faction research to student satisfaction research by stating that 


The application of theories of work organizations ... must be carried out with care. 
The primary difficulty with such applications is that they make the implicit assump- 
tion that higher education organizations are essentially the same as those in the world 
of work and, therefore, that one can think of students in those organizations as one 
would of workers in factories or offices. While the analogy might be stretched to fit 
faculty and staff, it is doubtful that students would see themselves in the same light as 
would workers generally. Though the analogy of worker productivity to student per- 
formance is especially appealing ... we must be careful not to push such analogies too 
far. (p. 377) 


Student satisfaction research has not been explicitly predicated on the 
models of worker satisfaction. Most studies of student satisfaction are 
descriptive research. One of the most thorough analyses of accumulated re- 
search about students in American colleges and universities was conducted 
by Feldman and Newcomb (1969) in the late 1960s. After reflective con- 
sideration of what was known at that time about the impact of higher 
education of students, they wrote that 

Our best guess at the moment is that a college is most likely to have the largest impact 

on students who experience a continuing series of not-too-threatening discontinuities. 

Too great a divergence between student and college, at least initially, may result in the 

marshalling of resistances. Too little might mean no impetus for change. From this 

point of view, a college’s objectives might include a tolerance, or even a desire, for 

those discrepancies that can stimulate change and growth. (p. 332) 


Studies of student satisfaction have continued in this best-guess-at-the- 
moment approach. In these descriptive studies, satisfaction has been 
defined in various ways. For example, Aitken (1982) and Babbitt and Bur- 
bach (1985) take the term to mean student acceptance of academic 
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programs and living conditions. Bean and Metzner (1985, p. 523) have 
described satisfaction as “the degree to which a student enjoys the role of 
being a student and reports a lack of boredom with college courses.” A fur- 
ther example is provided by Nafziger, Holland, and Gottfredson (1975, p. 
132) who defined a satisfied student as one who is a “typical student at his 
college and [has] a personality pattern which is both consistent and well 
defined.” 

Astin (1974, 1978), after examining overall student satisfaction in a na- 
tional sample of university students from 194 American institutions, con- 
cluded that students’ overall satisfaction with university could be measured. 
Astin also identified specific facets of satisfaction that could be measured 
including teaching, curriculum, facilities, career preparation, extracurric- 
ular activities, and administrative services. He found that little variance in 
student satisfaction could be accounted for by entering characteristics, in- 
stitutional characteristics, and measures of student involvement (1974, p. 
168). Specific aspects of the college experience (ranked in Astin’s findings) 
which did have substantial effects on satisfaction were (a) quality of in- 
struction; (b) contacts with faculty and fellow students (Pascarella, 1980, 
has suggested that faculty and peer contacts have the greatest positive ef- 
fects that add significantly to overall satisfaction); (c) curriculum; (d) col- 
lege administration; (e) reputation of the institution (this effect is greatest 
among students with higher grades); and (f) intellectual environment. 

Willingham (1985) reported on a nine-institution longitudinal study re- 
lated to success in university. These universities and colleges, with the ex- 
ception of one California college, were all located in the northeastern 
United States. The purpose of the study was to identify precollege charac- 
teristics that were reliable predictors of success in undergraduate programs. 
He found that each institution had individualized definitions of student suc- 
cess. Generally, three types of success were recognized: academic, leader- 
ship, and significant personal accomplishment. Characteristics of successful 
students were determined from studying a group of students identified by 
faculty, administrators, and other students as the most successful students 
on each of the nine campuses. That study found that the precollege charac- 
teristics which correlated most highly with a student’s subsequent iden- 
tification as successful by college faculty at the end of an undergraduate 
program were (a) high school rank, (b) SAT scores, (c) high school honors, 
and (d) indications of follow-through in work or activities. 


Research Project 
A study of college transfer students attending the University of Alberta was 
conducted during the 1987-88 winter session (Vaala, 1988). In this study, 
“transfer program” was defined as a university-level program of study that 
commences with enrollment at a college and culminates at a university 
where the remaining courses required for a specific degree are completed. 
“Transfer experience” was defined as a period of transition that begins 
when a college student initiates application to the university and ends when 
that student personally identifies as a university student. “College” was 
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defined as a nonuniversity, postsecondary educational institution offering 
university-recognized, university-level credit courses. 


Purpose and Design 


The purpose of this study was to use student perceptions to identify, 
describe, and categorize those variables which are most closely associated 
with success and satisfaction in transfer from college study to a university- 
based degree program. Transfer students’ perceptions were separated into 
background factors and outcome factors in order to examine possible rela- 
tionships between perceptions of satisfaction and measures of success in the 
transfer program experience. 

A conceptual framework of the major theoretical elements of the study is 
presented in Figure 1. Relationships were assumed to exist between selected 
student characteristics (e.g., educational goals, career goals, student hous- 
ing, work experiences, previous academic performance, and individual at- 
tributes) and aspects of students’ transfer programs and transfer exper- 
iences. A relationship also was assumed to exist between selected supports 
(e.g., family, peers, and staff) and the transfer program. A further relation- 
ship was assumed to exist between a transfer program and transition from 
college to university as moving from one campus to another is inherent in 
the concept of a transfer program. Perceptions of satisfaction and success in 
the transfer program and experience were seen as self-determined criteria 
of students. While a relationship between satisfaction and success was as- 
sumed, this was not viewed as causal. Satisfaction and success were seen to 
be overall outcomes of both the transition and the transfer program. 


Participants 


From a cohort of 1,081 transfer students at the University of Alberta, two 
random samples were drawn: (a) 400 students received questionnaires in 
October 1987, and (b) 100 students were invited to participate in an inter- 
view. The response rate for usable questionnaire returns was 67.3%. To 
clarify and enrich interpretations from the questionnaire data, interviews 
were conducted in January 1988 with all 10 students who volunteered to be 
interviewed. 

The ratio of female to male respondents to the questionnaire was higher 
than the ratio between the sexes in the population for this study (3:2 versus 
1:1). The average age of these students was 23.5, with a range from 17 to 
47. A large majority (82.9%) of the respondents was single. The mean high 
school matriculation average of these students was 70.9%. Although stu- 
dents transferred to university from a total of 14 Alberta colleges (four Ed- 
monton colleges and 10 colleges throughout other areas of Alberta), over 
60% of the respondents transferred from one non-Edmonton and two Ed- 
monton colleges. 

The interview participants resembled the questionnaire sample with 
respect to sex, age, degree program, type of college attended, and location of 
college attended. 
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Figure 1: Conceptual framework of the relationships among variables involved in 
the transfer process 


Data Collection and Analysis 


A 64-item questionnaire was developed for the study. Demographic data 
and information about attendance patterns were collected with 36 items re- 
quiring a choice from a list or a short written response. Data about goals 
and satisfactions were collected by 18 items with five-point Likert-type 
scales. There were six items about sources of assistance and goals also re- 
quiring participants to choose a response or write a short response. Finally, 
four items about transfer problems and preferences invited participants to 
provide longer written responses. 

An interview schedule addressing five broad areas was developed follow- 
ing initial analysis of questionnaire data. Two areas focused on experiences 
and definitions of satisfaction and success. Two areas were about experien- 
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ces in adjusting to university after transfer. The fifth area was about use of 
supports and assistance during transfer. 

In addition to examining the results for the total sample, several group- 
ings of respondents were developed as a means of exploring relationships 
between respondent characteristics and major variables in the study. These 
groups were formed on the basis of attendance patterns, college location, 
college type, degree program, and date of first university registration. 
Findings 
The major findings are summarized in five parts: (a) satisfaction with the 
transfer experience, (b) satisfaction with the transfer program, (c) success 
with the transfer experience, (d) success in the transfer program, and (e) 
use of supports and assistance by students in transferring. 

Satisfaction with the transfer experience. A majority of respondents 
(58%) reported that they were satisfied with the overall process of transfer- 
ring to university. Students who attended university transfer colleges and 
university/technical colleges (i.e., colleges offering only university-level 
courses and colleges offering university-level as well as vocational/technical 
courses) tended to report greater satisfaction with the transfer experience 
than did those who attended technical colleges (i.e., colleges offering only 
vocational/technical courses). 

Students tended to define satisfaction in the transfer experience as 
receiving university credit for courses taken at college and reaching the 
stage of feeling comfortable at the university. A large majority (83.7%) of 
students reported some difficulties in completing the transfer process, with 
the most frequently cited being adjusting at university to (a) larger class 
sizes, (b) less personal professor-student interactions, and (c) heavier aca- 
demic workloads. 

Satisfaction with the transfer program. Students most satisfied with the 
transfer program were those who attended a university/technical college for 
two or four winter terms, transferred at least 27 credits to university, and 
then attended university continuously for at least three full winter terms. 

A comparison of student perceptions of satisfaction at college and 
university with five aspects of the transfer program is presented in Table 1. 
The results of a correlated ¢-test reveal a statistically significant greater 
satisfaction was expressed with four of these items at college than with 
these same items at university. Students tended to report higher satisfac- 
tions from interactions with faculty and peers at college than with similar 
interactions at university. Students also tended to report more satisfaction 
with their intellectual development at college than at university. Positive 
faculty influences on students were perceived to be greater at college than 
at university. Generally, students were satisfied with decisions to attend 
both college and university. 

Further, satisfactions with facets of the transfer program correlated 
more highly at college than at university. Figures 2 and 3 present Pearson 
correlation coefficients among comparable aspects of the college and univer- 
sity experiences. With respect to both college and university, the highest 


177 


L.D. Vaala and E.A. Holdaway 


Table 1 
Means of Respondents’ Perceptions of Satisfaction with Five Aspects 
of the Transfer Program at College and University 


nn nnn ee eae tdtttEUEIdIEEIS EEE SSESSSSS 


Location of student experiences 


Aspects College University 
The student friendships | developed 4.2 O:/3°% 
have been personally satisfying 

| am satisfied with the extent of my 4.03 3.86* 
intellectual development since enrolling 

Faculty members with whom | have had 4.13 Sere fli 
contact are genuinely interested in students 

My interactions with faculty members 3.70 3.34** 
have had a positive influence on 

my career aspirations 

| am confident that | made the right 4.23 4.32 


decision in choosing to attend 


Note: Five-point scale, 1=strongly disagree, 5=strongly agree 
*p'<.05,."") =.01 


correlations of responses tended to involve pairs which included one of the 
two items about faculty members. 

Overall, students tended to report high satisfaction with the college ex- 
perience. Only 26% of the students indicated that their present preference 
was to have taken all courses at the university. The remaining majority 
reported a preference for having attended college before university. 

Success with the transfer experience. Most students perceived that they 
were able to obtain adequate information about transfer while at college. 
Students most likely to hold this perception had attended a university trans- 
fer college or university/technical college full time for two or four winter 
session terms and transferred to university within a year of leaving college. 

About half the participants visited the university before transferring. 
These students tended to report such visits as beneficial to their transfer ex- 
perience. Also, before transferring, most students had some contact with 
persons at the university; friends were the most frequent contact. Technical 
college students were the least likely and university transfer college stu- 
dents were the most likely to have friends already registered at university. 

Most students reported that they felt at ease and had developed an iden- 
tity as a university student by the end of their first winter term at univer- 
sity. Students tended to perceive the major adjustments to university as 


including (a) larger classes, (b) heavier workloads, and (c) less personal 
professor-student interactions. 
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Figure 3: Pearson correlation coefficients among satisfaction with facets of the 
transfer program at university 
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Success in the transfer program. Respondents tended to report a decline 
in grades from college to university, with the mean decline in grade point 
average being 0.5 on a 9-point grading scale. This overall decline appeared 
unrelated to sex, degree program, college attended, or attendance patterns. 

Students who attended university transfer colleges or who attended col- 
lege full time for two or four winter-session terms tended to transfer the 
equivalent of one or two years of study toward their university degree. 
These students were also most likely to carry and complete a full-course 
load (i.e., 15 credits per term for three winter session terms). 

Using the measures of credits completed and grade point average, stu- 
dents who were more successful in the transfer program tended to (a) 
report higher personal educational goals, (b) perceive that their earnings 
went to pay their educational costs, (c) report a greater preference for a 
transfer program, and (d) have parents who were more likely to have com- 
pleted a postsecondary program. 

Use of supports and assistance. Students tended to use sources of assis- 
tance differently at university than at college. They frequently mentioned 
the positive value of college faculty influences, particularly in program and 
career planning. At college, friendships with peers were generally a source 
of social and academic support. College registrars were described as more 
helpful than college counselors in providing information about transfer 
programs and transferring. 

Difficulties in the transition from college to university were reported by 
83.7% of the students (Table 2). Over half these students reported more 
than one difficulty. With few exceptions, all difficulties were related to 
situations encountered in Edmonton or at university. Nearly a quarter of 
the reported difficulties (21.6%) related to the personal adjustments of leav- 
ing one social environment and entering another. The largest category of 
difficulties (30.7%) related to the academic aspects of the transfer program. 
Slightly less than a quarter of reported difficulties (23.1%) related to the 
physical surroundings of the university, while almost a quarter (24.6%) 
were about administrative services or procedures related to the transfer 
process. 

Over half the students commented that increased college and university 
attention could be given to providing students with current and accurate in- 
formation related to transferring. By contrast a minority of students indi- 
cated that the college and the university did not need to provide new or 
additional assistance for students who transfer (34.2% and 21.2% respec- 
tively). 

Most students described using a different form of support system at 
university than at college. University faculty members were perceived less 
frequently than college faculty members as providing assistance in program 
or career planning. University administrative faculty members (e.g., assis- 
tant deans) tended to be perceived as more helpful than were administra- 
tive staff (e.g., receptionists and clerks). Peer contacts at university were 
generally described as useful for sharing academic information; social 
friendships were frequently a continuation of friendships formed at college. 
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Table 2 
Frequency of Difficulties Faced in Transferring to University 
(n=216) 
ee ee ee OE Re ee ae OF TR WS 
Category By category Overall 
Difficulty f %F WF 
Personal (21.6) 
Forming new friendships 25 24.5 pe) 
Moving to a new city 17 16.7 3.6 
Financing educational costs 17 16.7 3.6 
Loss of contact with family/friends 14 13.7 3.0 
Finding accommodation 13 12.7 2.8 
Arranging transportation 5 4.9 Ast 
Leaving employment 4 3.9 0.8 
Other 4 6.8 1.4 
Academic (30.7) 
Larger class size at university 39 26.9 8.3 
Heavier workload requirements 28 19.3 5.9 
Lack of interaction with professors 21 14.5 4.4 
Greater academic competition 17 AAS 3.6 
Readjustment to academic life ilps) 10.3 3.2 
Adjustment to a new grading system 10 6.9 21 
Writing Competency Test requirement 6 4.1 123 
Adjustment to new teaching methods ‘o) 3.4 VA 
TOFEL requirements 4 2.8 0.8 
Campus environment (23.4) 
Impersonal nature of the campus 38 34.9 8.1 
Overwhelming campus size 33 30.3 7.0 
Unfamiliarity with facility locations 27 24.8 5.7 
Unfamiliarity with locating materials fi 6.4 1.4 
Rapid pace of activity/events 3 2.8 0.6 
Parking 1 0.9 0.2 
Administration (24.6) 
Lack of assistance to plan program 33 28.4 7.0 
Loss of course credits 29 25.0 6.1 
Late official acceptance 14 12.1 3.0 
“Red tape” 12 10.3 ae 
Requirements for applications 9 7.8 2.0 
Registration procedures 9 7.8 2.0 
Other 10 8.6 2.1 
Total 485 100 


Note: Some students provided more than one response. 
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Family members, choice of residence, and financial assistance were seldom 
mentioned as influential forces in program decisions or program comple- 
tion. 


Implications 
The transfer student study has several implications for practice as well as 
for theory and further research. 


Context for the Future 


The future direction of postsecondary education in Alberta rests in part on 
the increased availability of publicly accepted transfer program oppor- 
tunities. Long-range planning at the University of Alberta, the second 
largest university in Canada, projects a staged decrease in undergraduate 
enrollments in order to accommodate an increased institutional emphasis 
on graduate programs (University of Alberta, 1987). Similarly the Univer- 
sity of Calgary, a large and growing Canadian university, has been gaining 
increased national and international recognition as a research-oriented 
university. As these major universities move to limit undergraduate enroll- 
ments, other institutions in the Alberta postsecondary system are expand- 
ing their undergraduate programs. 

Establishment of the Alberta Private Colleges Accreditation Board in 
1983 (Government of Alberta, 1984) has enabled three private junior col- 
leges to attain four-year degree-granting status. In their role as colleges af- 
filiated with the University of Alberta, these colleges previously admitted 
substantial numbers of transfer students. For selected degrees, students are 
now able to complete all undergraduate work at these colleges, which would 
serve potentially to lower the number of transfer students moving to 
universities. However, the new degree-granting status of these colleges may 
also serve to expand further public awareness and acceptance of Alberta col- 
leges in general as “stepping stones” to university study. 

Further, despite provincially imposed funding restrictions on the expan- 
sion of programs at Alberta public colleges, an Edmonton community col- 
lege was authorized in early 1988 by the Department of Advanced Edu- 
cation to establish university transfer courses for several hundred transfer 
students, effective September 1988. While this move may also have been in- 
tended to alleviate concerns of overcrowding at the University of Alberta, it 
too may contribute to further enhancement of the public’s acceptance of 
transfer programs. Thus, in an era when the transfer function of colleges is 
being actively questioned in the literature (Cohen, 1985; Kissler, 1982; 
Knoell, 1982), postsecondary education policy in Alberta appears to be 
strengthening rather than diminishing transfer opportunities for students. 
Consequently, colleges will have to adjust to the systemwide trend to accept 
more postsecondary students in transfer programs. 

There are implications from this study for two broad areas of transfer 
programs: faculty-student interactions and transfer-program information 
sources, 

Faculty-student interactions. Faculty members appear to contribute con- 
siderably to student decision making about program and career choices, 
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particularly at the college level. This activity should continue to receive in- 
stitutional recognition and support. The positive outcomes of faculty-stu- 
dent interactions with respect to program advising and career planning in 
the college setting should be widely acknowledged and encouraged. Sup- 
ports and assistance should be routinely made available to individual faculty 
members to ensure than they have current knowledge about transfer 
programs. Further, university faculty members may need to conspicuously 
identify ways and means of ensuring that transfer students are accorded 
opportunities equal to those of other students with respect to interacting 
with individual faculty members about concerns over program and course 
information. 

Transfer program information. Transfer students have identified the 
positive value of visiting the university campus before transfer and of 
making contact with administrative faculty in appropriate academic depart- 
ments. In many instances, individual students have initiated these ac- 
tivities; in other cases, college staff have coordinated these events. These 
present forms of contact should be strengthened and encouragement 
provided for college and university staff to develop increased opportunities 
for transfer students to visit the university campus before commencing 
courses at that location. 

Although many students were found to be aware of the Alberta Transfer 
Guide (Alberta Council on Admissions and Transfer, 1987), they held 
diverse interpretations about the guide’s purpose. To resolve many student 
concerns related to transfer of credit, copies of the guide should be made 
available at college to individual students. Some difficulties in transferring 
have arisen when students were not given direct access to the guide. In ad- 
dition, the guide should clearly and conspicuously inform its readers about 
how it is to be used. For example, some difficulties have arisen when stu- 
dents generalize from the acceptance of a specific college course for a 
specific university degree program to a conclusion that the course is also ac- 
ceptable for other university degree programs. 


Implications for Theory and Research 


Further transfer student study should focus on the process of student 
decision making with respect to transfer program decisions. The finding 
that many students viewed a college experience as a preparatory step for 
university prompts a call for research to build on Zemsky and Oedel’s 
(1983) model of the structuring of college choice in order to address student 
anticipations/expectations when deciding to attend college before university 
and when deciding which university to attend. Zemsky and Oedel’s perspec- 
tive of the elements that influence college choice may identify important 
dimensions to include in a student decision making model. Additional study 
at this juncture could add valuable insights to the literature about college 
choice in areas such as student-institution match and institutional response 
to changing student demographics. 

Investigation also needs to move beyond examination of the functional 
outcomes of faculty advising (Hornbuckle, Mahoney, & Borgard, 1979; 
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Stickle, 1982) by developing credible perspectives on the mentoring/model- 
ing role of faculty in colleges with respect to student career and program 
decisions. (For example, what are the activity patterns of faculty with 
respect to assisting transfer students? Are faculty equally accessible to all 
transfer students? What factors are associated with effectiveness?) Volk- 
wein, King, and Terenzini (1986) have begun investigation of transfer stu- 
dent intellectual development by focusing on student-faculty interactions 
before and after transfer. In the same light, expanded attention should be 
given to identify the dimensions of college and university faculty influences 
on transfer students. For example, what is the scope, nature, and intensity 
of this influence with respect to choice of courses, choice of university, and 
the shaping of expectations about the nature of the university portion of the 
experience? 

Formulation of a transfer student decision making model also should in- 
corporate the nonacademic aspects of the college-university experience 
which Pascarella (1986) suggests students perceive as substantial influen- 
ces on their decision making processes. For example, this study identified 
some variables, such as parental completion of postsecondary education 
programs, aspirations for degrees beyond the bachelor’s level, and students’ 
intention/perception of paying their own education costs, that may be as- 
sociated with students who were more successful in the transfer program. 
Similarly, Willingham (1985) concluded that student success contained 
academic, leadership, and personal accomplishment elements. Although 
participants in this study seldom reported on the development of leadership 
skills, greater attention should be given to examine the influence of per- 
sonal accomplishments and leadership opportunities on transfer student 
decision making. 


Concluding Comments 

A study of transfer students assessed the effectiveness of several dimen- 
sions of Alberta college-based university programs. The substantial positive 
influence of college faculty members on transfer students is a reflection of 
the support and encouragement available to students within the college sys- 
tem. Lack of student recognition for other support services at colleges may 
suggest that these are taken for granted or not extensively used by transfer 
students. 

The effects of college socialization appear to have strongly influenced 
transfer students. For example, students tended to strongly affirm that the 
college contributed positively to their intellectual development. Compared 
to college, however, the university portion of the transfer program ex- 
perience produced a disproportionately large number of negative comments 
from transfer students. Although students “moved on” because they felt 
they were ready for university, they also appeared to find university sub- 
stantially different from what they had anticipated. 

This perspective openly questions the notion that transfer students em- 
brace the university portion of their studies. In the Alberta postsecondary 
system which appears to be placing greater emphasis on the transfer func- 
tion of colleges, university administrators need to pay close attention to cir- 
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cumstances surrounding transfer student satisfactions in order to ensure 
that enrollment pattern projections are actually achieved. Clearly, there is a 
need to involve more institutions and to continue a more detailed examina- 
tion of variables relevant to the transfer experience and of the students who 
participate in this experience. 
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EDITORIAL 


Readers of AJER will be saddened to hear of the recent death of Dr. S.C.T. 
(Stan) Clarke, who was a frequent contributor to and longtime supporter of 
the journal. In place of an editorial, Dr. Vern Nyberg kindly consented to 
write a brief note on Dr. Clarke’s active and productive life in education. 
T.O.M. 


Stanley Charles Tremaine Clarke (1912-1989) 


The death of Stanley C.T. Clarke on Monday, May 22 in the Royal 
Alexandra Hospital in Edmonton ended a long and distinguished career in 
education. This career spanned a full half century, and for much of that 
time he was one of the better known and respected educators in the 
province of Alberta. 

Clarke was born in Scrooby, England, a town situated a few miles east of 
Sheffield. In 1918, at the age of six years, he moved with his parents to a 
farm near Castor, Alberta. He attended high school in Castor, Alliance, and 
Alberta College, Edmonton, then proceeded to the University of Alberta 
where he completed his BA and MA degrees. Next he served as principal of 
Two Hills school (four years) and principal of Carstairs school (two years). 
In 1936 he married Eleanor Coote, who died two years ago. He is survived 
by two sons, William and Philip of Edmonton, a daughter, Susan of Anahim 
Lake, B.C., and four grandchildren. 

From 1943 to 1945 Clarke was a navigation instructor in the RCAF with 
a rank of flying officer. After his release from the air force he completed his 
MEd degree at the U of A, after which he was engaged as a teacher in the 
prematriculation school in Calgary, an institution where war veterans could 
upgrade their education. He left this position to attend Stanford University, 
and on completion of his doctorate in education was appointed Acting Head, 
Department of Educational Psychology at California State Polytechnic Col- 
lege. 

In 1950 Clarke returned to Canada and joined the Department of Educa- 
tional Psychology in the Faculty of Education at the University of Alberta. 
In 1959 he commenced a nine-year term as Executive Secretary of the Al- 
berta Teachers’ Association, after which he returned to the U of A as Direc- 
tor of Special Sessions. In 1977 he retired and was granted Professor 
Emeritus status. 

After his retirement, Clarke continued to offer a course, Psychology of 
Learning and Teaching at the Adult Level, on a regular basis for 10 years. 
He was active as a researcher in education until his death. 

One of Dr. Clarke’s many special contributions to education was his 
dedicated service to the Alberta Advisory Committee on Educational Re- 
search (AACER), and to its successor, the Alberta Advisory Committee for 
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Educational Studies (AACES), which to this day is one of the primary sup- 
porters of AJER. He was the first secretary of AACER, and later the assis- 
tant director. For 10 years he was a member representing the Alberta 
Teachers’ Association, and then, following his return to the University of 
Alberta staff, served as executive secretary until his retirement. Following 
his retirement, he was an honorary member of the committee. 

Publications by Clarke, besides many articles in AJER, include two 
textbooks, Youth of Tomorrow, and The Modern World of Work (with T.W. 
Woodsworth); and three larger reports on aspects of education in Alberta, 
Classroom Discipline, The Goals of Teacher Education (with H.T. Coutts), 
and The Future of Teacher Education (with H.T. Coutts). His final publica- 
tion for AACES, entitled The Development of the Alberta Advisory Commit- 
tee for Educational Studies: 1954-1984, was distributed in 1986. His easy 
writing style caused him to be in demand by various organizations when 
special reports and historical accounts were needed. 

Awards related to education that were given to Clarke included: Kappan 
of the Year by the University of Alberta chapter of Phi Delta Kappa; 
Honorary Life Membership in the Alberta Teachers’ Association; Honorary 
Life Membership in the Canadian Education Association; and the Canadian 
Teachers’ Federation Special Recognition award. 

Above all, Clarke was an educator. Despite achieving positions of in- 
fluence and prestige, his primary concern was the classroom teacher whose 
task was the day-to-day delivery of education to the young people of the na- 
tion. 

Vern R. Nyberg 
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Ontario Institute for Studies in Education 


Denominational Cause and 
Employment in Religious Schools 


The topic of this article is the right of Roman Catholic and other denominational schools 
in Canada to discriminate on the basis of creed in the employment of teachers and other 
personnel. Three major questions are asked: What is denominational cause? How do 
statutes related to collective bargaining and the resolution of disputes affect denomina- 
tional rights? How are different groups of employees affected by the protection of 
denominational rights? Tentative conclusions are drawn, although the situation is in a 
state of flux. Implications are suggested for both denominational and nondenominational 
schools. 


This article examines the rights of Roman Catholic and other denomina- 
tional schools in Canada to discriminate on the basis of creed in the employ- 
ment of teachers and other personnel. The research is based both on an 
analysis of documentary evidence, including books, reports, laws, regula- 
tions, memoranda, and judicial decisions, and on interviews with officials in 
a number of provinces. Most of the court cases which were identified related 
to Roman Catholic schools, but these cases clearly raise questions concern- 
ing other denominational schools, privately or publicly financed, and may in 
time have implications for public nondenominational schools as well. 


Background 
Provision for the operation of denominational schools exists either formally 
or informally across Canada (Lawton, 1987). Publicly supported schools 
specifically for Roman Catholics are formally recognized in Ontario, 
Quebec, Alberta, Saskatchewan, Newfoundland, the Northwest Territories, 
and the Yukon. Informal accommodations have been made within the 
public system in New Brunswick, Nova Scotia, Prince Edward Island, and 
Manitoba for such schools, though this tradition has weakened in recent 
years. British Columbia makes no provision for separate schools for Roman 
Catholics within the public system: Catholic schools there exist as private 
institutions and receive (since 1977) some direct support from the govern- 
ment. Private denominational schools operate in most provinces and receive 
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government support to varying degrees in Quebec, Manitoba, Sas- 
katchewan, Alberta, and British Columbia. 

The existence of denominational schools raises a fundamental question: 
To what degree can denominational schools employ discriminating practices 
in order to maintain their denominational character? Increased public fund- 
ing of denominational schools has drawn more attention to this question. 
The Canadian Charter of Rights and Freedoms, ensconced in the Constitu- 
tion Act (1982), has further heightened the issue. The boundaries on accept- 
able practices are now in the process of definition through the courts. 

At present, the rights of employees in separate and denominational 
schools and the authority of trustees for these schools over their employees 
are matters that are by no means well defined in legal terms, though it is 
clear that, under the law, separate and denominational schools differ from 
nondenominational public schools insofar as the employment of staff is con- 
cerned. 

For any given province, at least six sets of “laws” are relevant to the 
issue: The Constitution Act (1867, formerly the British North America Act, 
1867); the Constitution Act (1982) including the Charter of Rights and 
Freedoms; provincial human rights codes; provincial acts governing the set- 
tling of disputes among educational employees; provincial or local collective’ 
agreements; and school board bylaws and regulations. Any one or all of 
these may come into play for a specific situation. 


The Issues 
At the risk of oversimplifying, three major questions are at issue in the 
relevant cases that have gone before the courts: (a) What is denominational 
cause? (b) How do statutes related to collective bargaining and the resolu- 
tion of disputes affect denominational rights? and (c) How are different 
groups of employees affected by the protection of denominational rights? 


Denominational Cause 


Denominational cause is the term used to describe the practice of dis- 
crimination in employment practices which is justified on the basis that par- 
ticular denominational attributes are essential for the performance of a 
specific job. In order to maintain the religious character of certain organiza- 
tions, their governing bodies claim the right to hire and dismiss employees 
on the basis of the candidates’ religion. For example, an Anglican church 
may discriminate against non-Anglicans and employ only Anglican priests. 
The notion that employees may be dismissed for just cause is often found 
in negotiated contracts and education acts (MacKay, 1984). The conclusion 
that actions related to the denominational nature of a religious school 
qualify as just cause seems a logical one. This conclusion, however, has not 
been easy to apply. Although a number of court cases involving denomina- 
tional and separate schools have been decided on the basis of whether a 
given reaction, such as the dismissal of an employee who has behaved in a 
particular way, is acceptable given the religious character of the school, the 
particular behaviors that can justify dismissal are not fully spelled out. Con- 


sequently, one is left to surmise which behaviors and reactions are accept- 
able and which are not. | 
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Perhaps the most common type of denominational cause involves a 
teacher who has married, without church permission, outside the faith. The 
Supreme Court of Canada, for example, held in Caldwell v. Stuart (1985) 
that adherence to a religious faith, including its rules regarding marriage 
outside the faith, can be a bona fide occupational qualification for teaching 
in a Roman Catholic school. Although the case was argued under the British 
Columbia Human Rights Code and the teacher in question was employed in 
a private Catholic school (British Columbia having no separate schools), the 
logic of the court would seem relevant elsewhere: 


The Board found that the Catholic school differed from the public school. This dif- 
ference does not consist in the mere addition of religious training to the academic cur- 
riculum. The religious or doctrinal aspect of the school lies at its very heart and 
colours all its activities and programs. The role of the teacher in this respect is fun- 
damental to the whole effort of the school, as much in its spiritual nature as in the 
academic. It is my opinion that objectively viewed, having in mind the special nature 
and objectives of the school, the requirement of religious conformance including the 
acceptable observance of the Church’s rules regarding marriage is reasonably neces- 
sary to assume the objects of the school. (Caldwell v. Stuart, 1985, p. 18) 


Other cases involving the question of marrying outside the faith have in- 
cluded Essex County Roman Catholic Separate School Board and Porter et 
al, (1978), Essex County Roman Catholic Separate School Board uv. 
Tremblay-Webster et al. (1984), and Walsh and Newfoundland Teachers’ 
Association v. Newfoundland (Treasury Board) and Federation of School 
Boards of Newfoundland (1988). The latter appears to be the first such case 
to reflect the use of Section 29 of the Charter of Rights and Freedoms, 
which reaffirms denominational rights (Anderson, 1989). 

A similar denominational cause arises in the case of unmarried couples 
living together. In Blatt v. Catholic Children’s Aid Society (1980), it was 
found that a Jewish man hired to work with disturbed teenagers could be 
fired because of his lifestyle, that is, his living with a woman without their 
having undergone a marriage ceremony. Blatt had claimed discrimination 
on the basis of marital status, but the courts found that he had no marital 
status. Although this case involved the Catholic Children’s Aid Society 
rather than a Roman Catholic separate school, Blatt’s position as a social 
worker appears to be comparable to that of a teacher. 

Another denominational cause is a change in religion. If a person who is 
hired as a Roman Catholic teacher by a Roman Catholic school system later 
converts to another religion, he or she may well be dismissed, according to 
interviewees in several provinces. A change in religion (from Roman 
Catholic to Salvation Army) was a factor in the Walsh case noted earlier. 
On the other hand, the same system might employ a non-Catholic and ask 
only that the individual adhere to the tenets of his or her own faith. In other 
words, different expectations are held for Catholic and non-Catholic 
teachers, expectations that may or may not be spelled out in their employ- 
ment contracts. 

As noted earlier, there is no clear definition as to what constitutes a valid 
denominational reason for dismissing or refusing to hire a person. Ad- 
minstrators interviewed in Catholic school systems emphasized that they do 
not “go on witch- hunts” and that if teachers run into problems, whether in 
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their marriages or other aspects of their lives, the school administrators try 
to help them resolve problems within the procedures of the Church. For ex- 
ample, they probably would help them to acquire an annulment in the case 
of a marital breakdown. 

While it is not stated as such in court decisions, what actually seems to 
underlie the concerns of denominational authorities is whether the person 
involved acknowledges and accepts the authority of the Church over his or 
her behavior, or if he or she refutes this authority in public action. In the 
latter case, the religious authority that the school is teaching the children to 
accept would be undermined. Thus matters handled in full privacy, such as 
the use of artificial birth control, would probably not arise as a cause for dis- 
missal in a Roman Catholic school system, whereas the public advocacy of 
such practices in contradiction of the Church’s teachings in the matter 
would be. 

While marrying outside the church is perhaps the best example of 
denominational cause which has been tested in the courts, at least in the 
case of Roman Catholic schools, not all cases involving this issue have in 
fact led to the automatic dismissal of a teacher. For example, in Essex 
County Separate School Board v. Tremblay-Webster et al. (1984), the court 
found that a teacher had a right to grieve a dismissal order and have the 
matter go to arbitration because the collective agreement in the board 
guaranteed such a right. That is, the court found that the collective agree- 
ment took precedence over the rights that the school board had under the 
Constitution Act (1867). This type of ruling brings us to the second issue, 
the way in which statutes related to collective negotiations and the settle- 
ment of disputes affect denominational rights. 


Collective Agreements 


In general terms, the courts have held to date that, in provinces which have 
separate or denominational schools, the province may not make laws which 
mandate the settlement of disputes between teachers and school boards 
that involve denominational matters. In particular, such legislation has 
been held invalid when used to review cases where teachers have been dis- 
missed for denominational cause. Relevant cases include Essex County 
Roman Catholic Separate School Board and Porter et al. (1978). 

On the other hand, as in Essex County Separate School Board v. 
Tremblay-Webster et al. (1984) and Stack v. Roman Catholic School Board 
for St. John’s (1979), the courts have held that provincial legislation requir- 
ing collective negotiations per se does not prejudicially affect the rights of 
separate and denominational schools. Furthermore, the courts have ruled 
that if school boards agree in such negotiations to a reduction in their 
denominational rights, then they must abide by the agreements because 
they entered into them voluntarily. 

Finkelstein (1985), for one, questions the distinction that the courts have 
made between provincially mandated practices and those entered into in 
voluntary agreements between boards and teachers. He points out that it is 
the class of persons holding the Catholic or Protestant faith and not their 
school boards that have their denominational rights in educational matters 
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protected by Section 93 of the Constitution Act (1867) and Section 29 of the 
Charter of Rights and Freedoms. He questions whether school boards have 
any more right to bargain away these rights than provincial legislatures 
have to legislate them away. While he may be correct, and the courts may 
reconsider this matter, in the meantime trustees for separate and 
denominational schools are probably taking greater care in their negotia- 
tions. In the Tremblay-Webster decision, the court noted that the clause en- 
suring a right to arbitration was inserted in the collective agreement after 
the earlier judgment in the Porter decision which stated that provincial 
guarantees for boards of references in dismissal cases did not apply when 
dismissal was due to denominational cause. 

Also, Newfoundland’s Catholic Educational Council has entered into a 
separate agreement with the Newfoundland Teachers’ Association to 
protect positions of members in religious orders, which had been threatened 
by the court’s decision in Roman Catholic School Board Exploits-White Bay 
and NTA (1983) that seniority rights in the contract could not be ignored in 
order to keep members of religious orders on staff. In this case the collective 
agreement was provincially negotiated in a secular setting and was ap- 
plicable to all school boards, whereas in provinces such as Ontario, collective 
agreements are negotiated locally and, in the case of Roman Catholic 
separate boards, are negotiated in a setting in which denominational rights 
are guaranteed by the Constitution Act (1867). Such a distinction can be 
important to a court that must interpret the intent of the agreement. 
Nevertheless, trustees of a Catholic school system who may be considering 
the possibility of laying off teachers who are not Catholics or who are non- 
practicing Catholics before laying off those teachers who are in good stand- 
ing with the Church would be well advised to ensure such action is legal 
under the employment contract that has been negotiated with their 
teachers. 


Employee Groups 


The final issue to be considered is the employment rights of those in dif- 
ferent positions. In simple terms, the courts have found that those not 
directly concerned with the educational mission of the school may not be 
discriminated against for denominational reasons (St. Paul’s Roman 
Catholic Separate School Board v. Huber et al., 1982). Thus religious re- 
quirements for custodians, secretaries, bus drivers, and cafeteria staff are 
not in order. Indeed, according to Hunter (1972) and discussed in Givan 
(1986, pp. 203-20, pp. 961-972), 


as a matter of policy [the Ontario Human Rights Commission] has declined to pursue 
complaints against separate school boards involving teaching positions, but has inves- 
tigated and attempted to conciliate complaints from applicants for clerical or 
janitorial positions who allege denial on the basis of their religion. (Hunter, 1972, p. 
241) 


At the same time, Roman Catholic separate school boards in Ontario 
may require all staff who are Roman Catholic to be separate school sup- 
porters. In Morra v. Metropolitan School Board and Callaghan (1981), the 
courts found that it was not religious discrimination for the board not to 
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hire as a custodian a Roman Catholic who had elected to be a public school 
supporter. Instead, it found that the Board had discriminated on the basis of 
economics; that is, the applicant’s decision to direct his taxes to the public 
system had cost the board revenue. Discrimination on this basis was not 
proscribed by the Ontario Human Rights Code. It might be noted that, had 
the applicant been non-Catholic, this possibility would not have existed and 
the Board could not have refused to hire the person whom they agreed was 
otherwise well qualified. 

A current dispute between the Ontario English Catholic Teachers’ As- 
sociation and the Metropolitan Toronto Separate School Board also hinges 
on the distinction between economic and religious discrimination. In this 
case, Gail-Anne Black, a Catholic, was terminated because she and her hus- 
band sent their children to a local public, rather than a Catholic, school: 


OECTA is contesting her dismissal on the grounds that the board’s policy requiring 
employees hired after Oct. 31, 1969, to be separate school supporters violates the 
Charter of Rights and Freedoms. They argue the policy is discriminatory because the 
Charter guarantees parents the right to educate their children in accordance with 
their conscientious beliefs. (Polanyi, 1988) 


Conclusion 

This article examines matters concerning employment in denominational 
schools: the definition of denominational cause, the role of collective agree- 
ments, and the effect of an employee’s role. The court cases which were 
analyzed led to the following conclusions. First, religion, with its inherent 
moral values, seems to be a bona fide occupational qualification for teaching 
in a denominational school. In practice, public acknowledgment of the 
religious authority of the church in question seems to serve as the standard 
basis for discrimination; public acts or statements which may appear to 
refute the values of the church seem to provide denominational cause for 
dismissal. Second, collective agreements have been interpreted by the courts 
as providing further security for individual employees (e.g., Essex County 
Separate School Board v. Tremblay-Webster et al., 1984). That is to say, 
school boards may negotiate a reduction in denominational rights and 
employees may receive additional safeguards concerning discrimination 
through a collective agreement. Third, discrimination for denominational 
reasons does not apply to all positions within the school system. Although 
denominational cause may provide just reason for discrimination concern- 
ing teaching personnel, it does not have the same validity when positions 
such as secretary or custodian are considered. 

Interpretation of case law is a complex task. Similar situations which 
seem to differ only in small ways may be judged differently. The cases which 
were presented here dealt with Roman Catholic schools. Yet increased 
public funding for denominational and other independent schools in Canada 
has brought these other schools into the public eye. Similar cases concerned 
with staff from these schools will no doubt appear before the courts in the 
future. 

Some questions which may arise include: How broad are denominational 
rights? Do they apply to minority Protestant sects? Fundamentalist 
religions? Secular humanists? What support would a religion proposing ra- 
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cial superiority receive? How will the questions which Finkelstein (1985) 
raises—that denominational rights belong to the people and cannot be bar- 
gained away by a selected group of trustees—be resolved? Clearly, 
denominational cause has been identified as relevant to teachers and not 
relevant to custodial and secretarial workers. Yet how does denominational 
cause apply in the employment or dismissal of principals, superintendents, 
psychologists, or consultants? 

Finally, because a Roman Catholic separate school board was able to dis- 
criminate against a Catholic who was not a separate school supporter, will 
public boards be able to demand that all their employees be public school 
supporters? 

These questions can in the end be answered only by the courts of law. 
For the present, teachers, trustees, and others should develop a keener un- 
derstanding of individual and collective rights. The more that can be done 
in contracts to refine the meaning and bounds of denominational cause, the 
greater control these individuals, and not the courts, will have over a ques- 
tion central to their lives. 
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Principals’ Evaluation of Teachers in a 
British Columbia School District: 

Are the Reports Professional or 
Bureaucratic Documents? 


This article reports an exploratory study. of school principals’ evaluative reports on 
teachers. The reports were drawn from three different times over a 10-year period in the 
recent past. The primary purpose was to determine whether the evaluation program which 
the reports reflected was bureaucratic or professional. In order to do so, it was necessary to 
identify through content analysis of teacher evaluation reports the criteria that school 
principals appeared to use when they evaluated the performance of teachers. Related to 
this main purpose were two secondary purposes. The first was to ascertain whether any 
change in the apparent criteria had occurred over the 10-year period. The second sought to 
compare the apparent criteria-in-use with what the research literature reports as desirable 
teaching practice. The study contained the assumption that a more professionally oriented 
program of evaluation would emphasize criteria that are research-based and that, over 
time, there would be evidence of a movement toward the use of such criteria and away 
from a nonresearch-based approach by those conducting the evaluation. The study found 
that the reports were neither professional nor bureaucratic documents. Rather, they ap- 
peared to serve ritualistic purposes that were of little help to the teachers’ professional im- 
provement interests and that were of little help to the organization’s decision making 
interests. Neither did the reports reflect their use of research-based criteria. Change over 
the 10-year period toward the use of more research-based criteria was only minimally evt- 
dent. 


Introduction 
When I begin with my graduate class in personnel administration in the fall 
term, two of the questions I always ask the students in the class are “What 
kind of evaluation have you had in the recent past?” and “What do you 
think of that experience?” Without fail, most of them report that they were 
evaluated infrequently, that when it did occur it was a rather superficial ex- 
perience, and that it was not a useful experience to them as practitioners in- 
terested in their “professional development.” The process may serve 
bureaucratic and political purposes; if so, they are prepared to put up with 
it. But as for serving a professional purpose, the evaluation experience was 
seen largely as a waste of time. Wise and Darling-Hammond (1984/85) ex- 
press a similar view based on a major study undertaken by the Rand Cor- 
poration reported in a monograph entitled Teacher Evaluation: A Study of 
Effective Practices (Wise, Darling-Hammond, McLaughlin, & Bernstein, 
1984). In the monograph, according to Wise and Darling-Hammond 
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(1984/85, p. 28), the authors suggest that in many school districts in the 
United States, “evaluation is a perfunctory, routine, bureaucratic require- 
ment that yields no help to teachers and no decision-oriented information to 
the school district.” 

According to these authors, all school systems treat teachers both as 
professionals and as bureaucrats and there is more or less tension between 
these treatments. Depending on one’s conception of teaching, one will per- 
form teacher evaluation differently. For the purposes of the study reported 
here, I distinguished between a more bureaucratic perspective as opposed to 
a more professional perspective on teacher evaluation. 

I argue that it is important to draw this distinction between professional 
evaluation and bureaucratic evaluation because local teacher evaluation 
systems are typically said to serve two major purposes: personnel decision 
making and professional improvement. In practice the former dominates. 
This domination is inconsistent with the literature’s emerging emphasis on 
evaluation practices which promote the professional development of 
teachers. If one wishes to emphasize evaluation practices which facilitate 
teacher professional development, one will attend to the growing body of 
evidence which appears to discriminate between effective and ineffective 
practices in teacher evaluation. 

In the study reported here, I assumed that a professional perspective on 
teacher evaluation is desirable in order to enhance the prospects of teacher 
improvement and growth. Whereas bureaucratic evaluation is characterized 
throughout a school or school district by a high degree of standardization of 
practice, a procedural orientation and the use of checklists when evaluators 
engage in data collection, the more professional approach is characterized 
by evaluation practices that are varied and flexible and that place an em- 
phasis on an analytic frame of mind toward criteria, kinds of evidence, and 
the interpretation of evidence. 

Using a bureaucratic approach, school districts typically rely on prin- 
cipals to evaluate teachers in a standard manner using general criteria. By 
focusing on these attributes in order to ensure reliability as much as pos- 
sible, principals limit the relevance and utility of evaluation for teacher 
professional growth. Wise and Darling-Hammond (1984/85) point out a 
number of problems that school districts face when they use this approach: 


The limits on their time and expertise and on the tools available to them mean that 
principals can, at best, assess whether the teachers in their charge are minimally com- 
petent ... bureaucratic evaluation at its best can identify teachers who lack minimal 
teaching skills. But the characteristics associated with effective bureaucratic evalua- 
tion make it irrelevant to the professional growth needs of the vast majority of 
teachers. For them it is a ritual. (p. 31) 


Whereas bureaucratic evaluation is meant to serve the needs of the or- 
ganization for monitoring how adequate the teacher’s performance is, 
professional evaluation is meant to help teachers meet their needs for sup- 
port and guidance relevant to improving classroom practice: “Thus, a 
credible evaluation process for assessing teachers’ relative competence and 
for improving teaching must attend to the particular teaching conditions 
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and demands that teachers face in their work” (Wise & Darling-Hammond, 
1984/85, p. 31, emphasis added). The point is, if real teaching improvement 
is to occur, more than short visits to classrooms by generalist observers is 
required where those observers largely base their collection of evidence on 
standardized checklists. Through their research and their findings, Wise 
and his associates have exposed an important condition with significant im- 
plications for those who are interested in and responsible for teacher 
evaluation and school improvement. The condition is the ritualistic nature 
of most teacher evaluation practices and the lack of utility of such practices 
in relation to teacher professional growth. 

For those who have an interest in improving the condition of teacher 
evaluation in Canadian schools, the following questions arise: What are the 
prevalent supervisory and evaluative practices in provincial schools and 
school districts? How ritualistic are such practices? To begin to answer 
these questions, in what follows I report the outcomes of a small, ex- 
ploratory study of factors used by school principals as they made judgments 
about teaching performance. These judgments are indicated in the evalua- 
tive reports written by principals in one British Columbia school district in 
the recent past. 


The Purpose of the Study 

The study reported here sought to enhance the quality of evaluative super- 
vision in schools by identifying the criteria apparently used by principals 
when they engage in the evaluation of teachers and write reports on them. 
“Criteria” was selected as the unit of analysis because they are one of the 
significant components of any program of evaluation. Anyone assessing a 
teacher’s teaching effectiveness must make judgments against a criterion or 
set of criteria (Storey & Housego, 1980). One of the ways of determining 
whether a program of evaluation is more or less bureaucratic is to examine 
the nature of the criteria used and the manner in which they are addressed. 

The primary purpose of this study, then, was to determine from an ex- 
amination of teacher evaluation reports by principals whether the evalua- 
tion was bureaucratic as opposed to professional. In order to do so, it was 
necessary to identify the criteria that elementary and secondary school prin- 
cipals appeared to use when they evaluate the performance of teachers. 
(Note: “appeared to use” is emphasized in the previous sentence because 
the analysis is based on the content of what principals write on official 
report forms. As a researcher, I was using evidence at least “once removed” 
from the actual situation.) 

Related to this main purpose were two secondary purposes. The first was 
to compare the apparent criteria-in-use with what the research literature 
reports is desirable teaching practice. The second was to ascertain whether 
any change in the criteria had occurred over a 10-year period. 


Research Design 
In order to discover the criteria (or factors) principals appear to use as the 
bases of their judgments on_ teachers’ performance, an analysis of 
principals’ teacher evaluation reports was undertaken. In addition, a litera- 
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ture search was done in order to ascertain what recent research says about 
teacher effectiveness criteria and desirable teaching behavior. 


The Research Questions 


This study sought to answer three questions: (a) What are the criteria prin- 
cipals appeared to use when they wrote reports on teachers in 1974/75, in 
1979/80, and in 1984/85? (b) How do principals’ apparent criteria compare 
with so-called desirable practice as reported in the literature? and (c) Have 
the principals’ apparent criteria changed over time, specifically between 
1975 and 1985? 


The Data Sources 


Officials in a school district in British Columbia were asked if I could have 
access to principals’ teacher evaluation reports. Access was granted on con- 
dition the executive of the district teachers’ association agreed. That agree- 
ment was sought and obtained. Forty-four reports were made available, 14 
from 1974/75, 15 from 1979/80, and 15 from 1984/85. Some of the reports 
were written by elementary principals and others by secondary principals. 
For purposes of maintaining confidentiality, the names of the teacher, the 
principal, and the school were removed from each report before I was given 
access to them. The reports represent approximately 20% of those written 
in each of the time periods. 

The reports were randomly selected from the available pool in the given 
year (1974/75, 1979/80, and 1984/85) by a senior official in the school dis- 
trict office. That official reported to me that five of the principals wrote 15 
of the reports (from two to five reports across the three time periods). 

By Ministry of Education standards, the district is considered midsized. 
It is located in the lower mainland of the province and has a rural as well as 
a suburban dimension. In 1984/85 there were 26 principalships, four of 
which were secondary. There were 445 fte teachers in the district working 
with 7850 fte pupils from beginning kindergarten through grade 12. The 
budget for the district was approximately $31 million. In 1984/85 there 
were five senior educational officials working out of the school district of- 
fice—a superintendent, two assistant superintendents, and two directors of 
instruction. The school board was made up of seven elected trustees. While 
there was no written policy governing the procedures and processes in- 
volved in teacher evaluation, senior officials expected that classroom obser- 
vation would be a major means by which evidence would be collected by 
principals. 

With respect to research-based criteria as reported in the literature, I 
chose to focus on Berliner’s (1984) review “which is framed as a review of 
factors that can be controlled or influenced by teachers and that are known 
to affect student behavior, attitudes, and achievement” (p. 52). Berliner’s 
four categories of factors are: 


1. preinstruction factors (content decisions, pacing decisions, grouping 
decisions, and decisions about activity structures); 
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2. during instruction factors (engaged time, time management, monitoring 
success rate, academic learning time, monitoring behavior, structuring, 
questioning); 

3. climate factors (communicating academic expectations for achievement; 
developing a safe, orderly, and academically-focused environment for 
work; sensible management of deviance; developing cooperative learn- 
ing environments); and 

4. postinstruction factors (tests, grades, feedback). 


These categories cover only some of the factors that might be considered. 
For purposes of this study, however, they provide a useful framework for 
comparing the factors or criteria principals used in the reports analyzed 
with the factors or criteria cited in the literature as important in terms of 
having an impact on student behavior, attitudes, and achievement. 


Analysis of the Data 


The standard form principals used to report on teachers is made up of three 
parts. The first identifies the teacher, the date, the school, and the school 
district. It also identifies information about the nature of the teacher’s ap- 
pointment, that is, grade(s) and subject(s) taught; kind of appointment 
(part-time versus full-time, temporary versus continuing); position held; 
and years of service. The second part of the report contains the display of 
evidence on which the principal makes his or her judgment about whether 
the teacher’s performance was satisfactory. In fact, in British Columbia, the 
report ends by law with a sentence that the teacher is judged as “satisfac- 
tory” or “less than satisfactory.” The third part of the report is the signa- 
ture of the evaluator. Prior to sending the report to the district 
superintendent of schools, the teacher sees a copy. In many instances, the 
teacher also signs the report. 

The first step in the analysis of the reports was to separate them into 
three groups by year (ie., 1974/75, 1979/80, 1984/85). The elementary 
reports were separated from the secondary, and certain of the demographic 
data were recorded (See Table 1). 


Table 1 
Number of Principals’ Reports By Level of 
Teaching Assignment and Gender of Teacher 


Time Period Elementary Secondary Total 
Male Female Male Female 

1974/75 6 4 2 2 14 

1979/80 2 if 4 2 15 

1984/85 4 5 3 3 15 

Total Numbers 12 16 9 te Ag 
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The second step was to take each report from a given school year and 
analyze the substance of the evaluation in relation to the Berliner categories 
(preinstruction, during instruction, climate, and postinstruction). Because 
all the factors or criteria used by principals in their evaluations did not coin- 
cide with the a priori classification, the category of “other” was added. 

This process of data reduction involved the following procedures: I took 
sentences in the reports and broke them into appropriate parts. For ex- 
ample, from report number two, 1975, I selected the following sentence, 
“Records are meticulously kept and the daybook, overviews, and previews 
are thoroughly prepared, indicating careful short term and long term plan- 
ning” and broke out these parts: meticulously keeps records; thoroughly 
prepares daybook, overviews, and previews; and carefully plans both short 
and long term. 

The third step in the analysis of the data was to make a judgment about 
each factor broken out as indicated above and accordingly enter it into one 
or another of the categories established for purposes of the descriptive 
analysis. In this case (see above), and using my knowledge of Berliner’s ex- 
planations of the factors that fit his categories, “record keeping” was 
entered under postinstruction factors, “preparation of daybook, overviews 
and previews” was entered under preinstruction factors, as was “long and 
short term planning.” 

When this kind of analysis was completed for each report in a given year 
(e.g., 1974/75), the fourth step was to consolidate the factors from all the 
reports under the same set of categories so that I had, for a given year, an 
overall picture of the factors that principals mentioned and the number of 
mentions for each factor. 

Factors were also classified as follows: I ignored those factors that were 
mentioned only once, recorded those that were mentioned more than once, 
and treated as important those that were mentioned in a majority (more 
than half) of the reports. 

The three sets of reports were analyzed using the procedures described 
above. The analyses were recorded in Table 2. An examination of Table 2 
enabled me (a) to indicate the factors that principals used when they wrote 
evaluative reports on teachers, (b) to compare the factors emphasized by 
principals in 1984/85 with those emphasized in the research literature, and 
(c) to compare the factors emphasized in 1984/85 with those emphasized 10 
years earlier and five years earlier. 


The Findings 
The findings are reported according to the three research questions. Table 2 
displays those factors (criteria) that appeared in two or more principals’ 
reports for each of the time periods selected for this study. The factors ap- 
pear down the left side of Table 2 under one or another of the five 
categories used in the analysis. Down the right side of Table 2 are columns 
indicating each of the three time periods from which reports were drawn 
(1974/75, 1979/80, and 1984/85). Across from each factor and under the ap- 
propriate year the number of mentions of each factor are recorded. The 
“important” factors (as defined in this study) are indicated by means of an 
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Table 2 
Factors in Principals’ Reports: 1974-1985 


Number of Times Mentioned 


1974/75 1979/80 1984/85 


|. Pre-instructional Factors: 


(General) planning” 
Daily planning” 
Long-term planning” 11 
Goals and objectives clearly defined - 
Prescribed curriculum followed - 
Sequence addressed in lesson - 
preparation 

Grouping practices Fi - - 
Uses cooperative activities in : : 2 
group settings 

Individualizes instruction 4 - 6 
Diagnostic knowledge of pupils’ 2 

cognitive growth levels 

Lessons related to students’ assessed - 3 : 
needs/interests 

Planned use of learning centres : 2 - 
Arranges “sponge” activities - - ” 
Arranges availability of extra : : 5 
resources 

Resources used to advantage - 2 - 
Uses resource persons in plans 


ONNMNN®, 


il. During-instruction factors 


Begins instruction with preview 


Begins lesson with clearly stated - 2 7 
objectives 

Engages in “whole group” introduction - - 4 
Uses teaching aids and devices 5 


Uses resources during teaching . 
Teaching techniques are varied : 
Uses questioning techniques 


ams , 
‘ 


(“important factors in Principals’ reports) 
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Table 2 (continued) 


Number of Times Mentioned 


1974/75 1979/80 1984/85 


Questions effectively 
Gives clear instructions 
Emphasizes verbal interaction 
Allows for pupil discussion 
Routinizes pupil movement 2 - 
Keeps students on task - 2 
2 


Cee 3° ee - eer) 
‘ 


Monitors and adjusts as necessary 
Attends to time management 
(transitions) 

Reviews and reteaches as necessary - 
Emphasizes “anticipatory set” 
Ensures all students enjoy some : 
success 


& ol. OO . 


‘ 
‘ 
mpm & W 


Ill. Climate Factors: 


Good rapport’ 14 1 


0 6 
Good classroom control* 14 6 8 
Good housekeeping” 12 ° 2 
Positive atmosphere, tone” 9 fe - 
Displays of students’ work” 6 2 8 
Teacher holds high expectations” 2 8 - 
Clearly stated expectations - ° 7 
Teacher shares ideas vs - - 
Teacher sensitive to students’ needs/ - 4 - 
emotions 
Uses positive reinforcement - - 5 
Students show good work and 4 4 - 


housekeeping habits 
Students show mutual respect 


IV. Post-instructional factors: 


Record-keeping (marks, attend.)* 
Marks students’ work and provides - - 11 
feedback” 

Checks seatwork 3 

Provides good assignments 2 - s 
Uses adequate grading procedures - 

Provides help to students with 2 

difficulties 

Reports to parents 
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Table 2 (continued) 


Number of Times Mentioned 


1974/75 1979/80 1984/85 


V. “Other” Factors 

Pleasing personality 

Teacher is cooperative 

Competence/skill 

Shows leadership 

Is strongly professional* 

Register, daybook well-kept 

Seeks/heeds suggestions for improvement 
Gives time to extra-curricular activities* 
Good relations with staff, office and/or parents 
Teacher has background knowledge 
Willing to share ideas 

ls a strong supporter of the school* 

Takes part in inservice activity 

Classroom has interest centres 

Uses furniture in flexible way 


Number of reports examined 


asterisk (*). The number of mentions, representing in each case a majority 
of mentions in each of the time periods, are indicated in boldfaced 
numerals, 


Apparent Principals’ Criteria 


In this section, I will summarize the criteria designated as “important” in 
each of the three time periods (see asterisked items in Table 2). 

1974/75. In the preinstruction category the reports emphasized “short 
term” and “long term planning.” No factors were emphasized in the during 
instruction category. In the climate category the reports emphasized “class- 
room control,” “good rapport,” “good housekeeping,” and “a classroom with 
a positive atmosphere and tone.” In the postinstruction category, “record 
ne, was emphasized. No factors were emphasized in the category 

other.” 

1979/80. “Daily planning” was emphasized in the preinstruction 
category. No factors emerged as important in the during instruction 
category. “Good rapport” and “teachers’ expectations for high pupil perfor- 
mance” emerged in the climate category. In the postinstruction category, 
only “record keeping” was emphasized. And in the category of “other,” 
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“taking part in extracurricular activities and intramural activities” and 
“good rapport with colleagues and parents” showed up as important. 

1984/85. The only important factor to show up in the preinstruction 
category was “planning.” In the during instruction category, no factors 
emerged as important. The climate factors which showed up were “class- 
room control” and “display of pupils’ work.” In the postinstruction category 
“marking pupils’ work and accompanying feedback” was indicated. Four 
factors showed up in the category “other”—“the teacher’s willingness to un- 
dertake extracurricular activities,” “the teacher’s support of the school,” “a 
strong professional attitude on the part of the teacher,” and “good rapport 
with colleagues and parents.” 

Discussion. One interesting feature of all the factors mentioned in the 
reports is that, for the most part, they are not operationally defined or em- 
pirically supported. It is not possible from the written reports to determine 
the evidence the principals used to make judgments about, for example, 
“good housekeeping” or “classroom control.” The judgments appear nearly 
always to be high inference judgments. The reports appear not to be profes- 
sional documents in the sense that they have not been written in a manner 
to facilitate professional growth. Neither do they appear to be bureaucratic 
documents in the sense that they do not reflect the use of standardized 
criteria from, say, checklists. Another feature appears to be the paucity of 
factors on which judgments were made. Still another is the apparent lack of 
emphasis on during-instruction factors. 


Principals’ Criteria: A Comparison Across 10 Years 

Table 2 displays “important” factors identified in the majority of reports in 
each of the categories and for each of the time periods. (Those factors are 
marked with asterisks.) Only one factor appears across the three time 
periods, namely “planning” (in the preinstruction category). No factors 
emerge as important in the during-instruction category in any of the time 
periods. Under the climate category, six items are recorded. Those that ap- 
pear two out of three times are “good rapport,” and “classroom control.” 
Those that appear one out of three times across the years are “display of 
students’ work,” “good housekeeping,” “positive atmosphere and tone,” and 
“holds high expectations for students.” In the postinstruction category, two 
factors emerged—“record keeping” and “marking students’ work with feed- 
back.” The former was important in two out of three time periods, the latter 
only in one. In the category of “other” there are four factors, none of which 
shows across the three time periods. Two of the factors show as important 
across two of the time periods (i.e., “giving time to extracurricular ac- 
tivities,” and “good relations with colleagues and parents”). The other two 
show up once across the three time periods. They are “the teacher is strong- 
ly supportive of the school,” and “the teacher is strongly professional.” 

Discussion. What does show up in the 1984/85 reports that was not 
present previously, is the use of terminology from the Hunter (1984) reper- 
toire (see Table 2). Note, however, none of the “Hunter factors” shows up 
as “important.” The use of the Hunter terminology is not surprising in light 
of the fact that, in this school district since the early 1980s, there has been a 
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districtwide staff development program for teachers and principals focusing 
on what is called “teacher effectiveness training” (TET). A clinical ap- 
proach to the supervision of teachers has also been part of this program 
(Grimmett, Housego, Moody, & Balasubramaniam, 1986). What is surpris- 
ing, however, is that not all the reports are replete with the criteria dealt 
with in the TET program, for example, criteria such as “anticipatory set,” 
“teaching to objective,” and “monitoring and adjusting.” 

The overriding impression from the 1984/85 reports is that they differ 
little from the 1974/75 and the 1979/80 reports in those factors this study 
labels as important. Only one factor, “planning” shows up across all three 
time periods. 

This is not to say that some reports in the 1984/85 period were not char- 
acterized by an emphasis on the TET language. The data appear to indicate 
that at least some principals were using the new language of pedagogy 
presumably learned through the districtwide staff development program. 


Research-Based Criteria and the 1984/85 Principals’ Reports: 
A Comparative Analysis 

In what follows, I compare Berliner’s (1984) researched-based factors with 
the factors that emerged in the 1984/85 reports. An explanation of the 
meaning Berliner attaches to his factors is provided and compared with 
what has been written in the reports. Comparisons are made category by 
category. All page-number references in this section refer to Berliner 
(1984). 


Preinstruction Factors 


Content decisions. Berliner defines what subject matter the teacher 
chooses to teach with the content-decisions factor. For example, even 
though the curriculum may call for the teacher to teach fractions, the 
teacher may choose not to do so. Berliner says, “the perceived effort re- 
quired to teach a subject-matter area, the perceived difficulty of the subject 
matter for students, and the teacher’s personal feelings of enjoyment while 
teaching a subject-matter area influence the teacher’s choice of content” (p. 
53). Content decisions could mean that student achievement in a given sub- 
ject-matter area simply will not happen because the student does not have 
the opportunity to learn in that area. 

A look at Table 2 suggests that the 1984/85 data are inconclusive. 
Whether the factor “planning” includes reference to content decisions by 
the teacher can only be surmised from the reports. In not one of them was 
there actual reference to evidence of teachers’ content decisions. 

Time allocation decisions. Such decisions have to do with the actual 
amount of time allocated by the teacher among and within the various sub- 
ject-matter areas. Research shows what Berliner refers to as “incredible 
eee in the time allocations that are made by different teachers” (p. 

Table 2 shows no direct reference to this factor. Again, one might argue 
that the factor of “planning” encompasses time allocation decisions. A close 
examination of the actual reports indicates no references to time allocation 
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as such, although references to daily planning and unit planning may take 
this factor into account. 

Pacing decisions. The more the teacher covers, the more the students 
seem to learn. What is significant about this point has to do with the 
variability across classes. “The choice of pace, like the choice of content, and 
the decisions about time to be spent learning particular content areas, 
determines student achievement” (p. 55). 

Not one of the 1984/85 reports makes reference to the factor of pacing. 
Again, however, those who wrote the reports may have asked questions of 
teachers about this factor, they may have examined long-term plans which 
enabled them to infer pacing, but if so it is not clear in the reports. 

Grouping decisions. Because the classroom is a collection of persons, 
teachers have to make decisions about student work groups. The decisions 
have to do with the size and composition of particular groups for particular 
purposes in particular subject-matter areas. Research indicates that par- 
ticular kinds of grouping decisions can affect positively the achievement 
levels of students (pp. 55-56). 

Table 2 provides some evidence that principals were attending to 
teachers’ decisions about grouping. In a minority of reports, references were 
made to group settings and individualizing instruction. However, there is no 
direct evidence in the reports themselves that principals examined docu- 
ments for or spoke to teachers about their grouping decisions. 

Activity structure decisions. The way in which student activities are 
structured will affect instruction, either enhancing or limiting the teacher’s 
ability to instruct. Examples of activity structures are lecture, discussion, 
recitation, reading circle, computer-mediated instruction, television, seat- 
work (p. 56). 

Table 2 suggests that in two out of 15 reports (1984/85), there was refer- 
ence to teacher planning “sponge activities” to engage students who had 
completed their assignments and needed something worthwhile to do 
during the remainder of the lesson period. An examination of the reports, 
however, shows little direct evidence that principals attended to this factor 
in their evaluation of teachers. 


During Instruction Factors 


Engaged time. This refers to the use of allocated time by students in 
productive ways. What researchers call “engagement rates” vary widely 
across classes. “Teachers need to be aware of engaged time rates—for in- 
dividual students and for the class as a whole—in order to ensure that a suf- 
ficient amount of time allocated to instruction in a content area is used by 
students in productive ways” (p. 57). Table 2 shows the during-instruction 
factors that appeared in principals’ reports in 1984/85. One factor at least 
implies a focus on “engaged time”; that is, reference to what Hunter (1984) 
calls “active participation.” Six out of 15 reports made reference to “active 
participation.” When a teacher seeks active participation, he or she is at- 
tempting in a variety of ways to keep students on task. Teachers behaving 


in this mode are attempting to maximize engaged time. 
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Time management. This refers to transition times within and between 
lessons. Much instructional time can be lost by poor time management prac- 
tices during transitions (pp. 58-59). Table 2 indicates that in four out of 15 
reports, attention was paid to this factor. 

Monitoring success rates. The more success students have at reasonably 
challenging tasks, the higher their achievement level. “Success rate ... ap- 
pears to be another powerful variable with known effects on achievement. 
Like other classroom variables, it needs to be monitored, evaluated, and 
often modified” (p. 59). Table 2 indicates that this factor was referred to in 
two out of 15 reports. The references were general—that teachers ensure 
that students enjoy some success. The reports provide no evidence that 
those who wrote them were specifically interested in particular students’ 
success rates in various subject-matter areas. 

Monitoring students working on their own. Monitoring in this sense is 
defined as behavior on the part of the teacher that keeps students who are 
working on their own engaged in their assignments, that is, keeps at a high 
level the number of substantive interactions between student and teacher 
(pp. 62-63). Five reports out of 15 mention monitoring (see Table 2). Ac- 
cording to the reports, teachers “monitor and adjust while teaching.” While 
Berliner’s reference is to monitoring students when they are on their own 
doing assignments, the reports refer to monitoring and adjusting while ac- 
tively instructing. The reports seem to be addressing something other than 
what Berliner is getting at. 

Structuring. This factor has to do with teachers communicating clearly 
to students their objectives and their directions (p. 63). Table 2 indicates 
that in seven out of 15 reports principals focus on “lesson objectives” being 
clearly stated. Two reports mention “giving clear directions.” This factor is 
addressed in the 1984/85 reports, although it appears in a minority of the 
reports. 

Questioning. Questions may be categorized as higher or lower order. 
Lower order questioning “promotes participation, establishes a factual data 
base ... and provides high success experiences for [some] students” (p. 64). 
For other students, higher order questioning leads to higher student 
achievement (p. 64). While Table 2 shows that in six out of 15 reports ques- 
tioning was mentioned, an examination of the reports indicates that no 
mention was made of levels of questioning. When the reports say “questions 
effectively,” one is left to wonder what “effective” is. 


Climate Factors 


Communicating academic expectations for achievement. This involves 
teachers setting high (academic) performance standards for those they 
teach (pp. 66-67). Table 2 shows that seven out of 15 reports included refer- 
ences to clear statements of teachers’ expectations. From the reports it is 
not clear, however, that the references are to academic expectations. 

Developing a safe, orderly, and academically focused environment for 
work, This factor involves the consistent and fair application of clear and 
reasonable rules. Also it has to do with teachers behaving in a businesslike 
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manner as they go about their work (p. 67). Table 2 indicates that three fac- 
tors appear to fit Berliner’s meaning—what is called in the reports “good 
classroom control” (8/15), “a well-organized classroom” (2/15), and 
“routinized student movement” (2/15). Note, however, only one of these 
factors is found in more than half the reports; that is, classroom control. 

Sensible management of deviancy. According to Berliner this factor is 
evident when teachers “nip in the bud” behavioral problems; when they are 
able to manage more than one thing at a time as they teach and deal with 
students; when they use “signals” for academic work [a means of monitor- 
ing involvement]; when they maintain “momentum” and “smoothness” in 
their presentations; when they emphasize “accountability” of students 
during recitations and assignments; and when teaching is characterized by 
“variety” [in methods and materials, etc.] (pp. 67-68). A number of factors 
displayed in Table 2 suggest that the sensible management of deviance is 
evident in the teaching on which the reports have been written. The follow- 
ing seem to be related: “good classroom control” (8/15), “friendly environ- 
ment” (6/15), “uses positive reinforcement” (5/15), and “students show 
mutual respect” (3/15). On the other hand, an examination of the reports 
shows that none of the items in Berliner’s review is referred to as such. In 
other words, the reports do not contain thick descriptions of teacher be- 
havior that would lead one to judge that a classroom is characterized by 
“good control.” So again one must ask, “On what grounds are the factors in 
the reports based?” 

Developing cooperative learning environments. When teachers develop 
cooperative learning environments they organize and conduct the work stu- 
dents do in such a way that every member of a group is required to par- 
ticipate in activities that can only be completed successfully “through 
interdependent and cooperative behaviour” (p. 68). Table 2 contains no ref- 
erence to this factor. None of the reports contains mention of students 
engaged in assignments involving “cooperative learning.” 


Postinstruction Factors 


Tests. Teachers engage students only in those tests which accurately 
reflect what has actually been taught (pp. 69-70). Table 2 shows that the 
reports contain no reference to teachers’ tests as such. One might assume 
from the factors which are reported that tests form part of the basis for 
teachers’ assessments of students, but there is no direct evidence to suggest 
that principals checked out and compared the tests teachers gave against 
what those teachers actually taught. 

Grades. Here Berliner is referring to the judicious use of grades that are 
tied to objective performance (p. 70). Table 2 shows that in two out of 15 
reports, principals made reference to the use by teachers of “adequate grad- 
ing procedures.” Whether this means that principals assessed student per- 
formance in relation to the grades assigned by teachers is not clear from an 
examination of the reports. 

Feedback. This factor has to do with the substantial use of corrective 
feedback in academic areas, valuing students’ contributions by using their 
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ideas, and contingent praise for proper behaviour (p. 71). Eleven out of 15 
reports (see Table 2) contain reference to feedback—“the teacher marks 
students’ work, provides feedback.” Again, however, one is hard put to 
know exactly on what the principals are basing their judgments. What do 
they mean by “feedback?” 


Discussion 
Table 3 demonstrates the relationships between Berliner’s research-based 
factors and the factors that appear in the principals’ reports for 1984/85. 
Berliner’s categories and the factors under each category are listed on the 
left side of Table 3. Down the right are three columns labeled yes, no, and 
undecided. Opposite each of Berliner’s factors is entered the factors men- 
tioned in the 1984/85 principals’ reports and the number of times in the 
reports each factor was mentioned. An entry in the yes column indicates a 
clear connection between what Berliner is writing about with respect to a 
factor and what the reports say; an entry in the undecided column indicates 
that I am unable to say with certainty that what the reports record and 
what Berliner is writing about are the same thing. An entry in the no 
column indicates that the principals are silent on the Berliner category. 
Table 3 suggests that we cannot be sure that the principals in 1984/85 
were using research-based knowledge when they evaluated teachers’ class- 
room performance. The uncertainty stems from what might be called the 
scarcity of factors and the generality with which they are stated. As Table 3 
indicates, only in six instances are we looking at what, in this study, are 
called important factors. Interestingly, in each of the six cases the factors 
are in the uncertain column which means that they are written at such a 
level of generality that it is not possible to infer in what empirical and 
operational activities they are anchored. This being the case, and taking a 
rather conservative view, I would judge the reports in 1984/85 do not reflect 
a research-based knowledge of the criteria of teacher effectiveness. 


Conclusions and Recommendations 

This small-scale, exploratory study set out to discover the criteria that were 
used in teacher evaluation reports in a British Columbia school district at 
three points in time over a 10-year period, whether those reports provided 
evidence over time of changes in the criteria used, and whether the most 
recent reports appear to reflect a clear knowledge of research on teacher ef- 
fectiveness on the part of those who wrote the reports. The larger purpose 
was to gather evidence that would throw some light on whether the reports 
could be thought of as professional or bureaucratic documents. 


Conclusions 


What criteria do principals appear to use? If we focus on the latest set of 
reports, the following criteria show up as the important ones: (a) planning, 
(b) classroom control, (c) displaying students’ work, (d) marking students’ 
work and providing feedback, (e) giving time to extracurricular activities, 
(f) strongly supporting the school, (g) behaving in a professional manner, 
and (h) having good relations with colleagues and parents. 
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Table 3 
A Comparison of Factors: 
Research-Based vs. 1984/85 Reports 


RESEARCH BASED FACTORS 
Berliner Yes 


PREINSTRUCTION FACTORS 
content decisions 


time allocation 
decisions 


pacing decisions 


grouping decisions 


activity structure 
decisions 


DURING-INSTRUCTION 
FACTORS 


engaged time 


time management 


monitoring success rate 


monitoring 


structuring 


questioning 


FACTORS IN REPORTS:1984/85) 
No Undecided/Unclear 


Planning (11/15)’ 


Planning (11/15)* 


Planning (11/15)° 


Individualized instruction 
(6/15) uses cooperative 
activities in group settings 
(2/15) 


arranges to 
have sponge 
activities 
available 
(2/15) 


emphasizes 
active par- 
ticipation 
(6/15) 


attends to 
transitions 
(4/15) 


ensures all students enjoy 
some success (2/15) 


monitors and adjusts while 
teaching (5/15) 


begins les- 
son with 
clearly 
stated ob- 
jectives 
(2/15) 
gives clear 
instructions 
(2/15) 


questions effectively (6/15) 
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Table 3 (continued) 


CLIMATE FACTORS 


communicates academic clearly stated expectations 
expectations for (7/15) 
achievement 


develops a safe, good classroom control (8/15)* 

orderly and academi- well organized classroom (2/15) 
cally focused environ- student movement is routinized 
ment for work (2/15) 


sensible management of good classroom control (8/15)* 
deviance uses positive reinforcement 
; (5/15) 
students show mutual respect 
(3/15) 


develop cooperative 
learning environment 


POST-INSTRUCTION 
FACTORS 


tests marks students work; provides 
feedback (11/15) 


grades ’ | uses adequate grading 
procedures (2/15) 


feedback marks students work; provide 
feedback (11/15) 


("Important factors in Principals’ reports) 
What is interesting about this list is the following: 


1. The lack of reference to any of what Berliner calls “during instruction” 
factors; 

2. The overall scarcity of factors; 

3. The high level of generality at which the factors are reported; and 

4. In relation to the first four, the last four are factors which fall outside 
the actual classroom. In other words, what principals value about effec- 
tive teaching is only partially related to what teachers do in classrooms. 


Did the principals’ apparent criteria change over time? The evidence from 
the reports suggests that the situation in 1984/85 was not very different 
from the situation in 1974/75. It is necessary to point out, however, that by 
1985 some principals were using a quite different language of pedagogy to 
express themselves in the body of the reports. The different language, used 
in a minority of reports in 1984/85, was likely an outcome of a districtwide 
teaching effectiveness staff development initiative in which both principals 
and teachers had been involved over the past few years. 
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Do these criteria reflect the current literature on teacher effectiveness? 
This question is difficult to answer because of the level of generality of 
statements in the reports and because, therefore, it is not possible to deter- 
mine the specific evidence that principals used to form their judgments. The 
paucity of factors also seems relevant to this question as does the evidence 
that no during-instruction factors made it into the important category as 
defined in this study. Taking a rather conservative view based on a close ex- 
amination of the reports, I concluded that the reports show little evidence of 
the application of research-based knowledge. (Note: this conclusion assumes 
that an objective, empirical, low-inference data base is required for the pur- 
poses of the school district’s teacher evaluation program if that program is 
concerned with the relationship between teacher evaluation and teacher 
development.) 

Looking now at the larger question about the nature of the reports and 
to what extent they reflect a bureaucratic orientation to evaluation in con- 
trast to a professional orientation, what can be said? Recall that a 
bureaucratic evaluation is characterized by a highly standardized approach, 
one that is procedurally oriented, and one in which the criteria are general. 
A more professional evaluation is characterized by a clinical approach 
focused on the details of practice and the joint analysis of that practice by 
the principal interacting with the teacher. 

On the basis of the evidence, what appears to be happening is a “diluted” 
kind of bureaucratic evaluation. While the format of the forms used for the 
official reports is standardized, the content of the reports is not. In other 
words, principals appear to take an ad hoc approach to report writing when 
it comes to the criteria they use to judge performance. They do not act on 
the basis of districtwide or schoolwide policy on an acceptable set of com- 
mon criteria. 

It seems quite clear that the reports are not professional documents. For 
reasons discussed above, one can argue that the reports form a weak foun- 
dation on which to engage in the clinical analysis of teaching. 

If the reports are neither bureaucratic nor professional documents, what 
are they? They appear to serve a bureaucratic purpose, that of enabling offi- 
cials to say they have complied with the requirements of the School Act in 
relation to the evaluation of teachers. Beyond that, the reports appear to be 
the product of a ritualistic act with little meaning from a professional 
development perspective for those who are involved. 


Recommendations 


Given the exploratory nature of this study and the limited number of 
reports, it is not possible to generalize the conclusions to other districts. 
However, the study has thrown some light on the question of criteria and 
the nature of the evaluation process in one district. It is possible to make 
some recommendations to officials and others in the district if responsible 
parties are interested in making the evaluation program more professional- 
ly oriented. Based on the conclusions of this study it is recommended: 
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1. That a policy be developed by teachers, principals, and other officials to 
establish at least a core of criteria to be used in the evaluation of 
teachers. The policy could be either school-level or district-level. 
Presumably the emphasis would be on research-based criteria. 

2. That the policy address the related problems of criterion measures, the 
expertise of those who evaluate teachers, the means by which evidence 
is collected and shared, and the time required to do the work entailed. 

3. That there be a designation by appropriate officials of the human, 
material, time, and fiscal resources required to accomplish the develop- 
ment and implementation of the policy indicated in (1) and (2) above. 

4, That there be developed a means of monitoring the implementation and 
continuation of the teacher evaluation policy in order to ensure it is in 
place and moving in the direction of a more professional approach to 
evaluation as opposed to a merely bureaucratic one. 


Postscript: Caveat Emptor 

This article suggests that the criteria used in judging teaching be research- 
based and not merely whimsical and ad hoc. Recently a point of view has 
emerged that questions the desirability of using research-based criteria 
when rating teacher effectiveness. Scriven (1987), Popham (1987), and 
Denham (1987) all express the view that research on teacher effectiveness 
cannot guide teacher evaluation, that objective evaluation techniques can- 
not guide in an effective way the process of teacher development, that the 
validity of any approach that rates teachers solely on the basis of the num- 
ber of performance criteria actually met is to be seriously questioned. The 
views of these authors, in other words, take exception with what has been 
called “scientific” or “deterministic” evaluation. Smyth (1989) argues for 
an approach to supervision and evaluation which is “educative” as opposed 
to “managerial.” 

In the face of such views, and with the point of view that evaluation is to 
serve the purpose of teacher development, I would continue to argue for 
what one might call a “modified [educative] research-based approach.” Ac- 
cording to McLaughlin (1986), 


Theory and experience suggest that teacher evaluation of another stripe can support 
teacher growth and development, strengthen the role of the principal and contribute 
significantly to the vitality and coherence of the school. (p. 162) 


McLaughlin reports that some school districts are moving away from what 
he refers to as a “deterministic process-product model of teacher evalua- 
tion.” In such districts principals are trained to observe in classrooms, to 
help teachers think through classroom problems, teacher-pupil interactions, 
and the structure of classroom instructional practices. He says, “Principal 
training framed in this model acknowledges the conditional nature of 
teacher effectiveness and focuses on individual teacher’s judgments and 
choices within broad and widely held categories for effective teaching” (p. 
162). What appears to me to be indicated in the above view is the use of 
various kinds of research-based knowledge on teaching effectiveness as sen- 
sitizing devices whereby the concepts are thought of as “primitive” and 
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heuristic when evaluators observe, describe, and discuss teaching with 


teachers in a helping or development mode. According to Bacharach and 
Lawler (1980), 


Despite the inherent vagueness of primitive concepts they often (1) reveal the com- 
plexity and multidimensionality of phenomena that might otherwise be treated in an 
oversimplified or unidimensional manner, (2) serve as integrative devices for analyz- 
ing seemingly disparate ideas, and (3) lead to more specific, well-defined ideas. (p. 14) 


This approach appears to reflect that proposed by Wise and Darling-Ham- 
mond (1984/1985). The study of effective teacher evaluation practices 
reported by these authors found that in districts where evaluation em- 
phasizes improvement as well as personnel decisions, the districts have 
developed and use more professionally-oriented evaluation strategies. That 
is to say, “They have increased the key resources for evaluation—time and 
expertise—by resisting the bureaucratic impulse to treat all teachers alike 
and by involving expert teachers in the evaluation process” (p. 32). 

Also, such districts are moving toward a participatory approach (teach- 
ers and administrators) to evaluation as distinct from the more traditional 
adversarial approach. Such an approach not only enhances teachers’ rights, 
but also their responsibilities. “The move toward negotiated responsibility 
contains the seeds of collaborative control over teacher quality; it creates a 
framework within which educators—teachers and administrators—can 
work together to improve the quality of their common professional work” 
(p. 33). What Wise and Darling-Hammond are suggesting is a radical depar- 
ture from present practice. They are emphasizing certain features that 
would be common to any revamped evaluation program: (a) the evaluator is 
a specialized expert, (b) a wide range of indicators characterizes the evalua- 
tion format, and (c) the criteria used rely on high inference variables and 
are susceptible to individualized applications (p. 32). 

Do such features characterize present practice in British Columbia 
school systems? Before we can fully answer this question we need further 
evidence. For example, who are the evaluators? What kinds of expertise do 
they have? What criteria are applicable and acceptable to the various 
stakeholders? How is evidence collected so as to ensure as much as possible 
its relevance and reliability? How is the evidence shared between and 
among evaluators and those evaluated? By means of what procedures and 
processes are evaluation practices conducted? How are evaluation policies 
and programs created, implemented, maintained, monitored, and changed? 

On the bases of answers to these and other questions, it would be pos- 
sible to describe contemporary evaluation programs and practices in various 
educational jurisdictions (school, school district, or province), to appraise 
the nature of the programs and practices in relation to the bureaucratic ver- 
sus professional distinction, and then to prescribe policy decisions and ac- 
tions to be taken in order to begin to close the gap between the reality of 
present practice and a more desirable state of affairs in relation to teacher 
evaluation and teacher development. 
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Cognitive Flexibility and Inflexibility in 
Principals’ Problem Solving 


This study examined aspects of the problem solving processes of elementary school prin- 
cipals. Guided by information processing theory and recent social-psychological explana- 
tions of expertise in solving “messy” problems, the study was intended to determine (a) 
whether principals’ problem solving would demonstrate some of the same attributes of cog- 
nitive flexibility and inflexibility found in the problem solving of those working in other 
domains, and if so (6) to illustrate the nature of such flexibility in the context of school ad- 
ministration. Protocol data of several types were collected over a two-year period from 22 
principals, Results identify such sources of inflexibility as cognitive errors, uncontrolled 
moods, and lack of responsiveness to opportunities in the problem solution. In each case, 
more appropriate strategies are identified. Differences in flexibility between expert and 
nonexpert principals are briefly reported. 


Contemporary studies of what principals do and descriptions of highly effec- 
tive practice have contributed substantially to the knowledge base required 
for school improvement. While we are now better able to describe what 
some effective principals do in some situations, we know practically nothing 
about how they come to act as they do. What is it that shapes their practice, 
and what meaning do they find in such practice? Until these questions are 
addressed, our ability to assist principals in becoming more effective will be 
severely restricted. 

Research reported in the paper is part of a larger series of studies explor- 
ing the problem solving strategies and thought processes of principals. This 
series is in the tradition of problem-solving research based on information 
processing theory and is aimed at clarifying differences between expert and 
nonexpert principals in a variety of problem domains. Building on the find- 
ings of two directly preceding studies (Leithwood & Stager, 1986, 1989), 
this paper reports the results of examining one dimension of problem solv- 
ing: flexibility (defined in detail below). For purposes of this study, the at- 
tributes of flexibility in principals’ problem solving were defined on the 
basis of research carried out in other domains characterized by relatively 
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unstructured and messy problems. Two general questions were addressed: 
Does principals’ problem solving demonstrate attributes of flexibility and 
inflexibility similar to those found in the problem solving of people working 
in other domains? Do principals who engage in overt practices judged to be 
highly effective or expert demonstrate greater flexibility in their problem 
solving than their less effective or less expert peers? Answers to these ques- 
tions provide a basis for helping to decide whether further inquiry about 
principals’ cognitive flexibility is likely to be productive. 

The choice of flexibility in problem solving as a focus for this study 
emerged most directly from our prior explorations of principals’ problem 
solving. However, similar conclusions about the importance of flexibility 
have been reached independently by those working in a variety of fields. For 
example, Scribner (1986) points out that flexibility in meeting changing 
conditions (and what Schén, 1983, terms “informal improvisation”) are 
well-documented aspects of practical intellect among dairy workers,,. as- 
semblers, and warehouse employees. Results of Schwenk’s (1988) review of 
research suggest that cognitive flexibility is central to successful strategic 
decision making on the part of senior managers in private corporations. As 
far as we are aware, however, only Glasman (1986), in addition to oursel- 
ves, has acknowledged its promise for understanding aspects of educational 
administration. Glasman asserts: 


The value of focusing on flexibility in judgment rendering should not be mysterious. 
After all, flexibility is required whenever there is uncertainty. What may be new to 
students of school leadership is that the work which Bruner and Simon began con- 
tinues until this day, and its focus in cognitive psychology is on judgment research 
(e.g., Kahneman, Slovic, and Tversky, 1982).... Current cognitive psychologists who 
work on judgment rather than choice are sending the message that judgment under 
uncertainty is central. It is high time that students of school administration admit it 
too, and set out to examine it rather than be ashamed of it. (p. 167) 


Framework 
The framework for the study is based on several reviews of research carried 
out in related areas of social cognition. Showers and Cantor’s (1985) review 
of research on social cognition provided the general outlines of the 
framework. This review examined the relationship between elements of 
motivation and cognitive processes central to problem solving effectiveness, 
as depicted in Figure 1. The utility of this conception for interpreting 


Motivational Elements — Flexible Cognitive Strategies “Appropriate” Action 


Goals - Responsiveness to situations 
Mood - Active control 
Knowledge - Multiple interpretations 


- Change in repertoire 


Figure 1: Showers and Cantor’s (1985) Conception of the relationship between sources of 
motivation and problem solving 
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evidence from our preliminary work and the comprehensiveness of research 
on which it is based were reasons for adopting Showers and Cantor’s con- 
ceptualization, with some adaptation, into our framework. 

Motivation, according to Showers and Cantor (1985), is a product of 
one’s personal goals, mood states, and the knowledge one already possesses 
relevant to the problem at hand. Each of these sources of motivation guides 
one’s interpretations of a problem and plans for how to respond to that 
problem. The relationship between these three sources of motivation and 
one’s response to a problem is mediated by dimensions of one’s thinking. 
These dimensions, within which Showers and Cantor define “flexibility,” in- 
clude: recognition and responsiveness to novel aspects of a problem; degree 
of active control over one’s thoughts and plans; how one interprets a situa- 
tion; and change in one’s knowledge, beliefs, and values as a function of ex- 
perience. Evidence reviewed by Showers and Cantor suggests that social 
problem solving is enhanced as one moves toward the flexible end of each of 
these dimensions. Flexibility involves: 


(a) adjusting interpretations in response to situational features; (b) taking control of 
[one’s] thoughts and plans; (c) seeing multiple alternatives for interpreting the same 
event or outcome; and (d) changing [one’s] own knowledge repertoire by adding new 
experiences and by reworking cherished beliefs, values, and goals. (p. 277) 


The bulk of the evidence from research on social cognition, however, 
demonstrates more passive, less flexible ways in which prior knowledge con- 
tributes to one’s interpretations and strategies. The inflexible end of the 
four dimensions of thinking include tending to cling to favorite interpreta- 
tions and not be responsive to situations; exercising little active control over 
moods, self-defeating cognitions, and dysfunctional strategies; being 
trapped by perceptually salient and cognitively available stimuli; and having 
schemas and stereotypes in one’s knowledge repertoire that are resistant to 
change. In applying the Showers and Cantor (1985) conception, we decided 
to concentrate only on the first three elements of flexibility and inflexibility 
because the fourth was concerned with a time frame too lengthy to capture 
in our study. In addition, although we used the terms goals and moods in 
the manner of Showers and Cantor, we took a somewhat different approach 
(explained below) to the role of knowledge, which is the third motivational 
element considered by these authors. 

Two additional research syntheses were used to further elaborate on 
selected aspects of the Showers and Cantor formulation. Nisbett and Ross 
(1980) provide evidence related especially to the matter of knowledge and to 
the third aspect of inflexibility—“being trapped by perceptually salient and 
cognitively available stimuli.” While they do not deny the importance of 
motivation in situations where problem solving appears unsuccessful, they 
believe that there are additional explanations. According to evidence which 
they review, problem solving failures often are “cut from the same cloth” as 
successes. That is, certain strategies are well adapted to deal with a wide 
range of problem situations (for example, recalling what you did the last 
time you encountered a similar problem) but become a liability when ap- 
plied beyond that range (for example, when the current problem has unique 
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features that must be addressed). Most of the evidence in support of this 
claim was generated in relation to problems which required some under- 
standing of the normative principles that guide the professional scientist’s 
formal inferences (for example, rules for deciding how “representative” is a 
sample of opinion data). 

There are two general kinds of tools that people use in interpreting 
situations or in making inferences about them by “going beyond the infor- 
mation given.” The first kind of tool, much studied by Kahneman, Slovic, 
and Tversky (1982), involves using two judgmental heuristics to make in- 
ferences—the availability and the representativeness heuristics. In the case 
of the availability heuristic, judgments about the likelihood of an event are 
based on the availability of past occurrences in one’s memory. Frequently 
occurring events are usually easier to recall than those which occur infre- 
quently, so availability is a good way of estimating probability. But this 
heuristic may also distort estimates of probability because other things be- 
sides frequency can increase the availability of an event in one’s memory 
(for example, recency or vividness). The representativeness heuristic is used 
to estimate the likelihood of some state of affairs (for example, poor quality 
of a teacher’s instruction) given knowledge of some other state of affairs 
(for example, the inattentiveness of a student during a lesson). While this 
heuristic is an essential cognitive tool, it can mislead, for example, when it 
restricts the problem solver’s attention to superficial causes of a problem or 
when it is used in conjunction with an incomplete understanding of cause 
and effect relations. The other kind of tools to be used in addition to heuris- 
tics are knowledge structures: theories, beliefs, and other representations, 
in the problem solver’s mind, of external reality. These knowledge struc- 
tures, while often valuable, can mislead to the extent that they are incom- 
plete or incorrect representations of external reality and to the extent that 
they preclude attention to the details of the actual object or situation at 
hand. 

In combination, then, the three research reviews discussed above 
provided a set of categories for recognizing instances of cognitive flexibility 
and inflexibility among principals, as well as suggestions regarding the cog- 
nitive and motivational factors which may influence them. 


Method 
Sample 


Twenty-two elementary school principals (one female, 21 male) took part in 
the interviews on which our analysis is based. They were selected from 
three boards of education in order to ensure that the unique context of a 
single school board did not skew the results in unknown ways. Six of the 22 
principals were designated as experts. No single method for arriving at this 
designation, of which we are aware, is sufficiently robust to be relied on ex- 
clusively. As a result, principals considered effective or expert in this study 
had to pass through two screens, each with its own compensating strengths 
and limitations. First, two central administrators from each board were re- 
quested, independently, to indicate which, of all principals in their boards, 
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they would recommend as highly effective. Only principals nominated by 
both administrators passed this screen. The strengths of this method are 
that (a) it is based on extensive data; (b) it requires agreement by two 
people who may have different experiences with the principals; (c) it is 
thoroughly grounded in the actual work of the principal in a wide variety of 
contexts; and (d) it allows for the application of alternative criteria as the 
basis for defining effectiveness. Weaknesses inherent in this method in- 
clude: (a) the informal, “clinical” nature of the data base; (b) the unknown 
reliability of those judging the principals’ expertness; and (c) the unknown 
basis on which a judge’s decisions were made. 

All principals passing the first screen were given an extensive interview 
keyed to a four-stage conception of growth in principal effectiveness or ex- 
pertise called The Principal Profile (Leithwood & Montgomery, 1986). 
These interviews were conducted by researchers who were familiar with the 
Profile but were not involved in other aspects of the study. The strengths of 
this method are that (a) the criteria on which judgment was made are ex- 
plicit; (b) the criteria were based on a synthesis of all relevant empirical re- 
search reported in the English language up to 1985; (c) the interview 
instrument used to collect the data had been developed and refined during 
the process of approximately 200 interviews conducted prior to its use in 
this study; and (d) formal steps were taken to ensure high levels of 
reliability among interviewers. The weaknesses of this method include those 
typically associated with self-report data: There may be a discrepancy be- 
tween what principals say they do and what they actually do. As Figure 2 
indicates, the two methods used in designating principals expert or nonex- 
pert compensated for one another’s weaknesses reasonably well. 

Those principals who not only were judged as highly effective by both ad- 
ministrators from their boards but who also obtained high ratings (3+ or 4) 
on the Profile as a result of the interview were designated as experts. Six 
principals met both of these criteria. The remaining 16 principals were con- 
sidered nonexperts in reporting results. 


Sources of Variation Reputation-Based Principal Profile- 
Judgment Based Judgment 


1. Amount of data on which 


judgment was based Extensive Modest 
2. Period over which 
data were collected Long Short 
3. Explicitness of criteria Implicit Explicit 
4. Validity of criteria Unknown Highly valid 
5. Reliability of data 
collection procedures Informal Formal 


nn ne EEEEE EEE anEn 


Figure 2: Strengths and limitations of two methods of determining principals’ effectiveness 
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At the beginning of data collection, the six experts had an average of 15 
years of experience in school administration, while the remaining 16 prin- 
cipals had an average of 17 years of experience. The experts’ schools had an 
average of 506 students, and four of these schools had vice-principals. The 
remaining principals’ schools had an average of 350 students, and five of 
them had vice-principals. 


Data Collection Procedures 


Protocol data were collected over a two-year period using three types of 
process-tracing methods (Hayes & Flower, 1983): retrospective reports, 
think-aloud methods, and stimulated recall. The last two of these methods 
have been shown to be reasonably free of the major threats to the validity of 
verbal reports (Nisbett & Wilson, 1977) including the distorted reporting of 
cognitive processes, incompleteness of description, and failure of respon- 
dents to rely on memory or to rely only on what can be retrieved from long- 
term memory. As Ericcson and Simon (1984) argue, both think-aloud and 
stimulated recall methods primarily draw on the contents of respondents’ 
short-term memory, and such memory for cognitive processes appears to be 
accurate. Retrospective reports, however, rely primarily on retrieval from 
long-term memory; their validity, therefore, is more suspect. 

The first interview in Year 1 included three sets of questions. The first 
set referred to 25 paragraph-length case problems which were encountered 
and prepared for us by principals. These problems were provided to prin- 
cipals in the study in advance of the interview, along with instructions to 
read the case problems and to sort them into separate piles in terms of 
similarities of the solution process they would use. Questions in this part of 
the interview concerned why the principals sorted the problems as they did. 
The second set of questions, somewhat open-ended in nature and followed 
by detailed probes, focused on principals’ reflections on their own problem 
solving, how it had changed with experience, and factors (such as values 
and beliefs) perceived to influence it. The final set of questions elicited in- 
formation about how principals selected, from myriad problems they en- 
countered, those to which they assigned time and priority (see Leithwood & 
Stager, 1986, for results). 

This first interview instrument and procedures for its use were pilot 
tested, in several iterations, with 12 principals and with 10 teachers en- 
rolled in a preservice principal preparation course. Data were collected from 
the 22 principals in the main study sample by a researcher who had not 
been involved in the interview leading to the Profile ratings and who was 
unaware of the reputational rating of the principals. Interviews averaged 
about 11% hours; they were audiotaped and subsequently transcribed. These 
interview records were then coded and content analyzed. 

The second interview in Year 1 explored differences in problem solving 
associated with variations in problem structure. All principals were asked to 
rank a set of brief case problems (see Table 1) in terms of how clear, at the 
outset, the course of action to be taken was. Then they were requested, 
using think-aloud procedures, to provide detailed solutions to two of these 
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Table 1 


Problems Presented to Principals 


Resource Staff Your board provides consultative services to support personnel 
such as subject coordinators, program consultants, social 
workers, and psychoeducational consultants. However, as 
there are increasing demands made upon teachers (relating, 
for example, to computer use or special education), there is 
more and more demand for the services of these persons. You 
want your school to make the most efficient use of the 
resource staff allocation. 


Principal Entry You have been assigned to a new position. The present 
principal, who is very highly regarded by staff, community, and 
students, is being moved to a larger school after only two 
years in his present assignment. The school community (staff 
and parents) are very displeased that a new principal has been 
assigned. They feel that the board has not considered their 
wishes. How would you enter this situation? 


School Consolidation All year, you have been a member of a committee concerned 
with an emotion-charged issue, that of consolidating six 
schools in your area into five. In April, the committee makes 
the decision that a neighboring school will be the one to close 
and that approximately one-third of its students should be 
attending your school next year. How would you prepare for 
fall? 


Primary Language Program Your school’s population is now 50% Chinese. It is found that 
most of the Chinese students entering kindergarten and grade 
1 have little or no English. You don’t know if the primary 
language program being offered is meeting the needs of all 
students as well as it might. 


Rumor in the Community Next year, your school will begin to have a French immersion 
program. Your trustee phones to say that there is apparently a 
rumor abroad in the community that the regular English 
program will be phased out. This is not true, but a widespread 
rumor will be disruptive. 


Setting School Objectives Your new school is one in which staff have never been involved 
in the setting of school objectives and are not apparently very 
interested in doing so. You have come to believe that it is a 
very important thing for staff to set school objectives and to 
evaluate them at the end of the year. 


hypothetical cases (the most and least clear or structured ones). Finally, 
they were asked to describe two problems from their own experiences which 
were similar in degree of clarity or structure to each of the hypothetical 
problems. 

In the second year, a subset of 11 of the original 22 principals was 
chosen (five of these from among the original group of six experts, and six 
of these from among the original group of 16 nonexperts) to participate in 
interviews concerning their problem solving in group situations. The 
reduced sample was caused by a combination of limitations on the resources 
available for analysis and the need to expand the amount of data collected 
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from each principal. For six principals, both a small- and a large-group 
situation were investigated; for the remaining five, only one (small or large) 
meeting situation was studied. In each case, a brief preinterview was con- 
ducted with the principal prior to a meeting situation; the researcher in- 
quired about expectations and plans with respect to a problem selected by 
the principal for the meeting. Then the principal taped, in the researcher’s 
absence, the group session in which the problem was considered. Finally, 
using a stimulated recall technique, the researcher and principal listened 
together to the group meeting tape. The tape was stopped whenever the 
principal wished to comment on, or the researcher wanted to ask about, the 
principal’s problem solving; their remarks were taped on a second tape re- 
corder. 


Data Analysis Procedures 


Transcripts were prepared for the 44 interviews conducted in the first year 
and for the preinterview and stimulated recall portions of the 17 sessions 
conducted in the second year. Results reported in this article are based on 
the analysis of these transcripts, along with the audiotapes of the meeting 
sessions. 

The specific data examined included principals’ interpretations of and 
comments on their problem solving; observed solutions to hypothetical case 
problems; solutions as remembered and described by principals to actual 
problems they had encountered; principals’ interpretations of and com- 
ments on their problem solving in group situations; and observations of 
principals in group problem solving situations. Given our research purposes, 
any one of these categories of data taken alone has certain limitations. Rely- 
ing only on solutions to hypothetical case problems would prevent our un- 
derstanding how principals clarify and interpret complex problems. As 
another example, there may be marked differences among principals in 
their willingness to recount the actual past situations in which they have 
been less than successful, and even in their ability to remember such situa- 
tions. The triangulation of a variety of methods allowed us to search for pat- 
terns common across various types of data and helped to compensate for 
these potential limitations. However, because the possibilities of bias in the 
data were still sufficiently strong, the analysis was primarily concerned 
with identifying and explaining instances of flexibility and inflexibility per 
se and only secondarily with a comparison between experts and nonexperts. 

We began the analysis of this large set of data by defining several 
categories of plausible cognitive or inferential errors, as such errors are 
defined in the work of Nisbett and Ross (1980) and others discussed earlier. 
This choice was based on considerable experience with our data as well as 
on a careful reading of relevant literature. These errors included: 


1. Overweighting vividness: Situations in which principals’ judgments ap- 
peared to be unduly influenced by the vividness and/or emotional inter- 
est of information. 
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2. Generalizing from a small or biased sample: Situations in which 
principals’ judgments appeared to be influenced by a bias in sampling 
while collecting information. 

3. Overuse and misuse of knowledge structures: Situations in which 
theories or schemas are depended on so heavily that principals fail to at- 
tend to details of the actual situation at hand, or where such knowledge 
structures are poor representations of external reality. 

Next, without knowledge of expert, nonexpert designations, two researchers 

read 30 of the first year’s transcripts and noted (a) instances where these 

errors apparently were made, and (b) instances where a “more appropriate 
strategy” was articulated and the error avoided. When both researchers in- 
dependently noted the same error or the same more appropriate strategy, 
the instance was recorded. Criteria used in judging instances of errors or 

more appropriate strategies were largely based on Nisbett and Ross (1980). 

For this study, then, correct strategies are primarily those that involve the 

use of a rule which formal scientists agree is appropriate in this situation. 

However, in several novel situations, the researchers, and in some cases two 

colleagues, made judgments based on a broad reading of the problem solving 

literature. The latter judgments are necessarily considered somewhat tenta- 
tive. 

At this point, one researcher continued with the analysis of the remain- 
ing 14 first-year transcripts and the transcripts and audiotapes from the 17 
second-year interviews by identifying both errors and more appropriate 
strategies. The researcher, still unaware of expert, nonexpert designations, 
also noted any other instances throughout the 61 interviews in which prin- 
cipals (a) appeared to be markedly responsive to specific aspects of the 
situation; (b) were clearly influenced by mood or aware of mood effects on 
problem solving; or (c) were clearly influenced by goals in problem solving 
or aware of their effects. 

The data were then summarized, first in terms of the occurrence of 
various instances of cognitive inflexibility and flexibility, and then the ap- 
parent influences on them. Only then were differences between expert and 
nonexpert principals examined. 


Results 
Results from the analysis of the 61 interviews are described in two sections. 
In the first and major section, the elements of inflexibility and flexibility 
identified for all 22 principals are presented (Table 2). In the second, con- 
trasts are briefly drawn between the six experts and the 16 nonexperts in 
the study. 


Elements of Inflexibility and Flexibility 
Errors 
The first instances of cognitive flexibility and inflexibility to be described 
are the more cognitive ones, that is those observed when categories derived 
from Nisbett and Ross’s (1980) work were applied. Unless noted otherwise, 
data reported in this and the two subsequent sections were based on all 
types of interviews. Table 2 summarizes three major categories of influen- 
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Table 2 
Elements of Inflexibility and Flexibility 


Inflexibility Flexibility 
Errors More Appropriate Strategies 
1. Overweighting vividness in setting priorities e deliberate planning and priority setting 


e conscious perspective on role of vividness 


2. Generalizing from a small or biased sample e encourages collective work from 
representative group 


3. Overuse and misuse of theories and schemas 
(a) Fail to see that a situation is unique or e deliberate vigilance for dangers 


different from others in the past 
e conscious of particular features of 


situation 
(b) Fail to determine actual causes of e deliberate awareness of need to search 
problem for nonsuperficial causes 
(c) Fail to see there are new problems facing e has opinion that there are new problems 
principals and can identify some 
(d) Fail to modify a single approach or e conscious awareness of need for 
strategy in light of situational features variability and flexibility in strategies and 
roles 
(e) Use theories or schemas that do not - e awareness of individual differences 


accurately represent reality 
e hold theories that encourage flexibility 
and openness 


Uncontrolled Moods More Appropriate Strategies 

e fail to examine alternative courses of action e recognize own limits in being able to 
solve some problems 

e focus On own negative mood e conscious control of mood 

Lack of Responsiveness More Appropriate Strategies 

e miss Opportunities e take advantage of opportunities in 
situation 

e fail to see other possible courses of action e vigilance for opportunities to meet goals, 


particularly those related to students 


tial errors (or inflexibility), along with the more appropriate strategies in- 
dicative of flexibility. 

Errors in the first category, overweighting vividness, were made by six 
principals; all involved in some way making poor judgments regarding the 
determination of priorities among the various problems competing for their 
attention. In two cases, principals spent an excessive amount of their own 
time, at work and at home, trying to solve an extremely difficult and intrac- 
table personal problem of a staff member. In the remaining four cases, prin- 
cipals set priorities in their problem solving strictly in terms of the vividness 
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of the problems. For instance, one principal thought that the only problems 
needing attention were the ones that were relatively vivid or highly ap- 
parent and stated that “I keep prioritizing. When there are no problems in 
the in-basket or walking into my office, I go and play with the kindergar- 
tens, do this and that.” 

There were several more appropriate strategies mentioned by a com- 
parable number of principals as ways of dealing with the potential error of 
overweighting vividness. Some expressed the importance of deliberate plan- 
ning and setting clear priorities as a way of avoiding the sort of reaction to 
vividness exemplified above: “I spend time on overall planning and analysis. 
I have learned not to jump and not to assume.” Good planning also was a 
method used to prevent vivid crisis situations or to avoid dealing with too 
much emotional information: “I just quite frankly don’t see a strong payoff 
in spending time and energy raking up these feelings.” 

Some had a deliberate bigger picture perspective on issues that could 
otherwise have been seen as vivid. One, “this is a very poor move, and even 
good teachers can sometimes make a mistake,” and from another, “I don’t 
worry so much because I’ve found that you can blow it the odd time.” 

Three principals made errors of the second type, generalizing from small 
or biased samples. Each error involved collecting information from a 
restricted set of those people who would have some association with a 
decision. In one case, the principal based his interpretation of and planning 
for an innovative program on limited and biased information provided by 
strong advocates of the program. In another case, the principal was aware 
that his information was biased because he had deliberately chosen a group 
of only like-minded staff to work with him in setting objectives. Despite 
this, he seemed unaware of the dangers of disregarding other viewpoints. 

A similar number of principals indicated use of a more appropriate 
strategy, that of strongly encouraging collective work from a representative 
group of staff, as a method for avoiding such bias. About his school cabinet, 
one principal noted: “They speak their minds. They don’t hold back and say 
just what they want you to hear.... They come forward with information. 
With sharing like that, things come forward I haven’t thought of.” 

By far, the most numerous cognitive errors displayed in the interviews 
had to do with the overuse and misuse of theories and schemas. As shown in 
Table 2, these are of five kinds. Fifteen principals made two or more of 
these errors, with some making several. A similar number of principals 
mentioned at least once a more appropriate strategy to prevent or overcome 
such errors; however, only six of them exhibited such strategies markedly. 

One fairly common error of this type is failing to see that a situation is 
unique or different from similar ones encountered in the past. In recounting 
difficult situations, several principals described cases where they had failed 
to detect dangerous features in a situation. For example, one failed to detect 
that a parent complaint was more serious than usual, and another failed to 
detect the real agenda of a caller. 

Another variant of this error is the tendency, observed in the responses 
of some principals to the hypothetical case problem, to assume obstacles 
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which are not necessarily present in a problem situation but may have been 
present in a similar problem encountered in the past. For instance, based on 
past experiences, one principal assumed there would be “anger and hate 
and emotion” in a situation where several parents were requesting that 
their children be placed in a particular class, and that there would be 
“strong fear and emotion” in a situation where a principal had to enter a 
school in which the previous principal had been popular. 

More appropriate strategies in this area included a deliberate vigilance 
for dangerous situations (“the ones that could blow up in your face”) and a 
conscious awareness of the need to attend to the specifics of a situation, 
“clearly listening to the situation, the problem concern, and making sure 
the entire professional staff understands we’ve got a unique situation,” or 
“T realized this was a serious variant of a familiar problem.” 

A similar, but less frequent, error was failing to determine the actual 
cause of a problem. Some of these errors, reported by principals noting that 
“hindsight is 20/20,” concerned initial misinterpretation of others’ motives 
in situations facing the principal, while others involved the use of superficial 
causal explanations. 

The more appropriate strategy in this situation involved a deliberate 
awareness that things may not be as they seem at first. As one principal 
said, “It was not totally clear. It was tough to know. You often don’t know if 
it’s little stuff that is the problem, or if that’s just an excuse for something 
major.” 

It also involves awareness that it is necessary to analyze consciously and 
thoroughly causes at the outset of problem solving. Another principal said 
“The principal has to hold people back from jumping to solutions ... there- 
fore let’s analyze it and let’s start looking at some causes.... Only then do we 
look at possible solutions and their ramifications.” 

Another source of inflexibility involves the failure to see that there are 
new problems facing the principal because overreliance on a theory or belief 
that there are none prevents the observation of such new problems. This at- 
titude or opinion was expressed by some principals when they responded to 
a particular question, in the first interview, regarding the existence of new 
problems. Typical statements were: “I don’t sit and think ‘How am I going 
to handle that problem?’ because I’ve been solving the same problem for 20 
years!”; “I’ve never found a problem that was unclear. My steps are usually 
right 100% of the time”; and “No brand-new problems come to mind. I 
sometimes say ‘I was doing this exact thing 20 years ago.” 

A more appropriate strategy with regard to this matter appears to be a 
conscious awareness of and vigilance for the new problems that face schools 
and principals. Principals who realize this indicate that “there’s always 
something new,” that “there are always new problems coming up,” and 
tend to be able to immediately back up this opinion with concrete examples: 


Although there are always some standard problems you deal with in a school ... I have 
found that problems have changed in that they are more complex. Education has a 
much different profile, we are under attack much more than we were, and there is a 
much greater accountability factor. Due to the complexity of society today—and educa- 
tion can’t divorce itself from it—life is more complex so education has to be more com- 
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plex.... I have to be more of a lawyer. I have to be more facile not just with the Educa- 
tion Act but with other bills that relate to that, that have an impact on that. Since 
knowledge is increasing at a phenomenal rate, therefore my job’s changing ... as cur- 
riculum changes. 


Another error or source of inflexibility involving overreliance on a 
knowledge structure is that of failing to modify a single approach or 
strategy in light of specific situational features. Principals who do this sug- 
gest, in speaking about their own practices, that they always use a par- 
ticular approach, often one that is extremely useful in many situations. 
However, they do not suggest that they are aware of situations in which 
such an approach is inappropriate. Most of the responses concerned with 
this matter came from questions in the first interview regarding principals’ 
perceptions of their own styles. Some involved a strong reliance on always 
staying within board policy or on a particular approach with which the prin- 
cipal feels confident. For example, one principal stated: “I like to gather a 
lot of information up front, and I like, I love the brainstorming process. I 
use it all the time.” Another explicitly rejected alternatives to his own com- 
monsense strategy: 


The actual turning to a set of designed problem-solving strategies doesn’t turn me on, 
doesn’t enter into the kinds of ways I solve problems. Although I know some of them, I 
tend not to use it. At PD sessions, when I pick up two or three things and one con- 
tradicts the other, I tend to go back to my commonsense experiential base and it 
works. 


More appropriate and flexible practices involve conscious awareness of 
the need for variability and flexibility in one’s practice: 


It’s important to know the people, their strengths and weaknesses ... being sensitive 
at the time to how close to overload they are, and when to ask them to give something, 
and when to not, when to back off, and when to share the burden around a bit. 


One principal pointed out the need to have a contingency plan because “as 
you get more experienced, you realize that almost anything can happen.” 
Particularly in the stimulated recall interviews, some principals dis- 
played flexibility by indicating that they were aware of the requirement for 
variability in the role they played in group problem solving. In some cases, 
this involved categorizing clearly the type of problem being considered: 
I suppose that I look at problem solving in four basic areas when it comes to present- 
ing a decision to staff. There are the “tell” decisions ... that would just be done. There 
is the “sell” type ... where I would have to sell my position to staff. Then there’s the 
“consult” area, where I would like to consult with the staff and then let them know ex- 
actly that I’m consulting with them but J have to make the final decision. And then 
the final area, which is the highest level I think, are the “share” areas, where I would 
outline the problem to them: here are the parameters but together we are going to 
share in the overall plan.... I have openly told my staff that if they see me confusing 
the terms, blow the whistle and let me know. 


In other cases, it involved changing one’s role in the group in response to 
what was occurring. One principal, in commenting on what was happening 
in a staff planning committee meeting, said: 


I also want to mention that it’s not up to me to focus the discussion. Often I sit back 
and become a participant, nothing more. People don’t look to me necessarily on some- 


229 


M. Stager and K.A. Leithwood 


thing like this to make a decision. They feel it’s just as much their responsibility and 
they will focus the discussion. There are things that the principal and the vice-prin- 
cipal do decide if there is not much to discuss, but at the staff planning committee, if 
it’s a proposal or concept that is really the responsibility of the entire group, I back 
out as chairman and it just carries on itself. 


Later in the meeting, the principal points out that he must play a different 
role: 
This is the principal talking. I have to talk about regulations. Now I’m not there as a 


participant but now I have to ask questions. She made a statement that I just couldn’t 
let go. As a principal, I’ve got something I have to be responsible for. 


The final kind of error regarding the inappropriate use of knowledge 
structures involves using theories or schemas that do not accurately repre- 
sent reality. The examples of inflexibility or error with respect to knowledge 
structures which we have outlined up to this point involve misapplication of 
particular schemas when the more cautious or critical application of the 
same schemas would have served the principal well. Nisbett and Ross 
(1980) point out that there are particular schemas—especially those involv- 
ing stereotypes—which are “so lacking in foundation and predictive value 
that they almost invariably serve the user badly” (p. 40). There were a num- 
ber of instances where principals made strong generalizations about their 
staff. For example, one said, “Teachers can’t take complaints; they’ll fall 
apart emotionally” while another said, “Teachers don’t like to open them- 
selves up and have people come into their classrooms.” 

Another suggestion of erroneous schemas came from the metaphors and 
analogies used by some principals. One regarded an educational system as 
analogous to a military organization; another referred to some students as 
“the cream” and others as “the dregs”; and a third spoke of his staff as “my 
horses ... I can’t go anywhere without them.” 

Examples of more appropriate schemas came from principals who dis- 
played a clear awareness of individual differences among staff members, of 
their competences and circumstances at any time. One principal stated that 
“people move at their own rates, and all the research shows that you cannot 
get all of them up to level X at the same time.” Another, in referring to a 
staff member who was experiencing some extreme personal stress, indi- 
cated that: 


There has to be a lot of understanding on my part and the staff’s. We all have to be 
special with her. I don’t treat everyone the same. Fairness is one thing, and consisten- 
cy is another, but understanding is probably more important. We are different and we 
all have our own situations, and I think you have to work with that aspect. 


This principal was aware not only of special needs of staff but also of their 
accomplishments, strengths, and philosophies of education; this was also 
evident from a number of comments that he made during the stimulated 
recall interview session. 

Another category of statements was much more compatible with cogni- 
tive flexibility and provided a contrast with the inflexible stereotyping 
described earlier. These statements included well-articulated beliefs or 
theories about the value of collective work in problem solving. Like the 
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negative, stereotype statements, these involved generalizations about all 
staff, but they were generalizations of a different quality. It seems plausible 
to suggest that principals who act on beliefs such as these will empower 
their staff members by increasing the possibility that most, if not all, will 
contribute to group problem solving. Principals who believe in the more 
negative generalizations (“teachers want to do their own jobs, unassisted, 
and principals do too”) seem likely, we speculate, to prevent even those in- 
dividuals who are interested in working on collective problem solving from 
doing so; their inflexibility may in a sense generate a self-fulfilling 
prophecy. Examples of the apparently more empowering beliefs are: “I have 
a strong belief that everyone has something to contribute. I believe in let- 
ting the person take the risk” and “I believe that people have a mindset 
toward doing things better. I make it clear to staff that if they can see a bet- 
ter way of doing things, let me know and my nose won’t be out of joint.” 


Mood-Related Inflexibility 


Instances of inflexibility in the previous section were those primarily in- 
fluenced by cognitive factors. Those described in this section were as- 
sociated with more motivational influences. 

There were several instances in which principals’ moods produced con- 
siderable inflexibility and failure to examine alternative possible courses of 
action as judged, after the fact, by the principals themselves. In some cases, 
this led to poor choices of action. In one case, a principal decided to write 
letters rather than seeing an irate parent in person, a choice he later 
regretted: “I didn’t want to see him again ... he kept harassing me ... con- 
tinuing to put pressure on me.... So I refused to see him in person.” 

In other cases, a concern with one’s own mood and emotion had dis- 
tracted the principal from more appropriate concerns and choices. One 
principal had a continuing concern with his own emotional state throughout 
the interviews, and made a series of comments such as, “I’m really not 
afraid of anyone, but I guess now I’m afraid of him,” “I’d enter the situation 
with apprehension,” “My stress level is pretty low, I have tremendous con- 
fidence, but if I have no solution, my stress level can really go up.” Another 
principal repeatedly spoke of his “frustration” with the staff. 

More appropriate strategies for dealing with mood influences were 
demonstrated by a considerable number of principals. As was the case for 
those concerned with not overweighting the vividness of certain informa- 
tion, a sense of one’s own limitations in the type and number of problems it 
is possible to solve appears to be helpful in avoiding emotions such as 
frustration. For example, “There are some problems you can’t do anything 
about. They are just too massive. It may have taken years to get that way, 
so how are you going to turn that around in a few hours?” 

A considerable number of principals were aware of the need to keep their 
moods under control in group problem solving: “You have to have a process 
where you can get people to air their concerns and then work through the 
concerns in a fashion that is as impersonal as possible.” 
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A smaller number indicated awareness of the need to keep their own 
mood under control when solving a problem alone. One said: 

A principal has to be a person who doesn’t panic particularly easily, because so many 

things can go wrong, nothing ever goes exactly as planned, no matter how careful... 

You have to keep thinking, all the while. That’s one of the things that makes a good 

principal, one who can keep thinking and readjusting his or her thoughts as situations 

arise. 


and another that: 


You have to know how to deal with things that are urgent and not panic, but use plans 
and click them in calmly. You know you don’t have to solve any problems yourself. 


Part of the more appropriate approach involved tackling problems with 
an air of calm confidence. One principal described his style: “As my wife 
said, ‘When a situation arises, you don’t seem to get all flustered ... you are 
cool, don’t get emotional.’” He went on to say: 


I don’t worry as much as I used to ... you know you can make a negative statement.... I 
have enough self-confidence that I think it’s better that I should speak up and say 
what I think than worry about whether someone will put an obstacle in my way. 


Goal-Related Flexibility: Responsiveness to Possibilities in Situation 


In the previous sections, the instances which were clearest in the analysis 
and reporting were negative occurrences, indicators of cognitive in- 
flexibility. In this section, the instances which are clearest are positive oc- 
currences, indicators of cognitive flexibility. 

Among certain principals, particularly in their remarks during the 
stimulated recall sessions, there were six clear examples of a phenomenon 
which we termed (after Hayes-Roth & Hayes-Roth, 1979) opportunistic 
planning, where the problem solver makes decisions that follow up on 
selected opportunities that present themselves in the situation or “takes ad- 
vantage of opportunities.” For instance, one principal who was dealing with 
the problem of improving staff resources for disciplining students solved 
two other problems in the process. First, he became immediately aware 
during the group discussion that his vice-principal had a serious personal 
concern with some facets of the discipline situation and made a deliberate 
point of discussing it with him alone very soon. Also, in considering the 
overall problem, he indicated the availability of a Ministry resource docu- 
ment for assisting with solutions and, while formulating a process for deal- 
ing with the discipline situation and introducing the document, he realized 
that the approach being developed would have great potential utility as a 
model for introducing the other documents which arrived in the school with 
great frequency. 

In addition to these situations in which principals demonstrated specific 
strategies for taking advantage of an opportunity presented, some prin- 
cipals displayed an explicit awareness of interpreting problem situations as 
opportunities. One pointed out the possibilities for “image building” in 
several hypothetical case problems and in several problems that he had ac- 
tually experienced. 
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This appears to be part of a more general vigilance by some principals. 
Particularly when faced with unstructured or unclear situations, these prin- 
cipals look for those specific aspects of the situation which will allow them 
to meet their goals, especially those to do with staff and students. These 
principals have a number of goals in mind and are extremely responsive to 
the possibilities in any situation for meeting such goals, especially their 
most important goals. For instance, one principal said, “Whatever it is we 
do, it has to be based on whatever we’re doing for kids. That belief helps me 
make decisions.... When I look at a problem, one of the first questions I ask 
myself is “How will this benefit students?’” 

Evidence of unresponsiveness was of two sorts. First, in the stimulated 
recall interview, there were two clear cases of missed opportunities which 
the researchers noticed (although the principal, listening also, did not). 
Second, there were several instances in which principals seemed to be so 
focused on their own narrow goal that they were blind to any other pos- 
sibilities in the situation. In contrast to a colleague who was well aware that 
a system which focused too much on avoiding mistakes is one which will be 
inflexible and have poor solutions, one principal described his approach to 
problem solving as “I know what they will support. You follow policies and 
procedures set down.” The most dramatic example of this narrowness and 
inflexibility occurred during a stimulated recall interview. The principal was 
so consumed with “getting his own way” in a decision that he failed almost 
entirely to hear the strong opposition of most staff. When he stated that he 
wanted to get “good” information, he meant that he wanted to get informa- 
tion that agreed with his. There were also cases in which the principal failed 
to be responsive to possibilities in the situation because of an absence of any 
clear goals. 


Expert and Nonexpert Patterns 
The main purpose of this article is to identify and illustrate instances of in- 
flexibility and flexibility in the practices of school principals; it is not 
primarily to focus on comparisons between experts and nonexperts in this 
domain. However, the marked differences in the patterns displayed by these 
two groups merits brief attention. 

Before considering these results, it should be noted that only some of the 
principals in both the expert and nonexpert groups were good informants, 
while about half of the group overall made few statements indicative of 
either inflexibility or flexibility. In general, the good informants seemed to 
be at the extremes of the experiness distribution. 

Among the expert group, four of the six principals clearly demonstrated 
a variety of instances of flexibility and the other two a smaller number of 
such instances; none of these six experts showed any instance of in- 
flexibility. Among the nonexpert group, five of the 16 demonstrated a 
marked variety of instances of inflexibility and no instances of flexibility. 
Another nine of the nonexpert principals demonstrated some instances of 
inflexibility and a few instances of flexibility. Of the remaining two nonex- 
perts, one indicated no instances of inflexibility and a few of flexibility, 
while the other showed some inflexibility but considerable attention to 
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flexibility. That is, the data were clearest, most informative, and least in- 
consistent for the expert and nonexpert extremes. 

In summary, experts avoided errors, controlled their moods, and were 
responsive to opportunities in the situation. Nonexperts, at least those at 
the least effective extreme, made errors, were unable to control their 
moods, and were unresponsive to opportunities available in the problem set- 
ting. 


Discussion 

This study found evidence for three main elements of cognitive flexibility. 
First, such flexibility involves a total avoidance of cognitive errors and an 
ability, noted by many authors (e.g., Morine-Dershimer, 1986; Nisbett & 
Ross, 1980; Schin, 1983) to make fine discriminations among details of par- 
ticular situations. Second, cognitive flexibility involves controlling one’s 
negative moods and approaching problem situations with an air of calm 
confidence. Klemp and McClelland (1986), in studying characteristics of in- 
telligent functioning among managers, found that the competence of self- 
confidence was absolutely essential and served to drive the other 
intellectual competences. Third, cognitive flexibility involves being respon- 
sive to the possibilites in the situation, affording a clear illustration of what 
Sternberg and Wagner (1986) would regard as practical intelligence. These 
authors use Neisser’s (1976) definition of “intelligent performance in 
natural settings ... as responding appropriately in terms of one’s long-range 
and short-range goals, given the actual facts of the situation as one dis- 
covers them” (p. 137). 

It should be noted that cognitive flexibility never involves being over- 
responsive or spineless. According to other data we have collected, expert 
principal problem solvers have core values, beliefs, and goals that are invio- 
late. That is, they display extreme cognitive flexibility, but they are always 
guided by a coherent set of values (Begley, 1988; Leithwood & Stager, 
1989). 

Our data indicate that at least several types of cognitive flexibility and 
inflexibility, found among other widely disparate groups, can also be found 
among elementary principals as they do their work. These data also il- 
lustrate the forms such flexibility and inflexibility may take in the specific 
domain of elementary school administration. Such findings are potentially 
significant for the improvement of administrative performance. For ex- 
ample, identifying errors, in the Nisbett and Ross (1980) sense, creates the 
possibility of designing specific training in the use of more appropriate 
strategies. As another example, principals might also be assisted in becom- 
ing more sensitive to their overuse of otherwise helpful heuristics. Several 
crucial questions must be answered, however, before such potential can be 
pursued. One question concerns the teachability of error avoidance and 
other aspects of problem solving. We are optimistic that this is possible, to 
some degree, and are presently attempting to do it in an experimental con- 
text. But others are much more skeptical, and there is virtually no empirical 
evidence bearing directly on the question. 
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In our view, a second question points to the major limitations of the 
present study; that is, our adoption and application of criteria for defining 
cognitive flexibility and inflexibility (including errors) without any direct 
evidence of the relationship between such criteria and the problem solving 
effectiveness of principals. The question is: Does flexible cognition, as 
Showers and Cantor (1985), Nisbett and Ross (1980), and others define it, 
actually make a principal a better administrative problem solver? Our 
modest attempt to address the question in this study involved briefly com- 
paring samples of expert and nonexpert principals. That comparison sug- 
gested a much higher incidence of cognitive flexibility among expert 
principals. But this evidence is correlational in nature and hence only weak 
support for the importance of such flexibility for principals’ problem solv- 
ing. Further research is needed to understand fully the role that cognitive 
flexibility plays in expert administrative problem solving. 
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The purpose of the study was to determine the impact of family form on perceptions of 
early adolescent children. The sample was selected from schools in middle socioeconomic 
status neighborhoods and consisted of 136 children from intact families, 42 children from 
one-parent families, and 34 children from remarried families. The various perceptions of 
children’s functioning were investigated by using internal measures completed by children 
(Culture-Free Self Esteem Inventory, Family Assessment Measure, Family Satisfaction 
Scale) and external measures completed by teachers and parents (Health Resources Inven- 
tory, School Report Grades). The data were analyzed using both traditional confirmatory 
and the more recent exploratory procedures. The results of the study indicated that teacher 
and parent ratings of children from intact families were more favorable than their ratings 
of children from one-parent and remarried families, but children’s perceptions of themsel- 
ves and their families were similar across family form. Furthermore, the results of the ex- 
ploratory data analysis suggest that it would be quite difficult for teachers to distinguish 
between children from the three fumily form groups without prior knowledge. It was con- 
cluded that family form is perhaps more important for adults than for children. Finally, 
the results point to the need for further research regarding the relevance of family process 
variables (interpersonal interactions among children and adults) to the functioning of 
early adolescent children from one-parent and remarried families. 


A topic of increasing interest and importance to family practitioners and re- 
searchers is the impact of family form on the functioning of children. The 
term form, as it is used here, refers to the type of family arrangement in 
which parents and children live: intact (biological mother, biological father, 
and children); one-parent (biological mother or biological father and 
children); remarried (biological mother or biological father, stepmother or 
stepfather, and children). The present study compares early adolescent 
children living in intact, one-parent, and remarried families according to 
their self-esteem (self-perceptions), how they perceive the processes or in- 
teractional aspects existing in their families (perceptions of family), 
children’s satisfaction with their families (perceptions of family), teacher 
ratings of children’s behaviors (teacher perceptions), parent ratings of 
children’s behaviors (parent perceptions), and children’s academic achieve- 
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ment (cognitive performance). These aspects have been selected for the 
study because: (a) they include children’s functioning in school and in their 
homes; (b) they provide both internal (child perceptions) and external 
(teacher, parent perceptions) indicators of children’s functioning; and (c) 
they provide indicators of cognitive, affective, and behavioral functioning. 

Many research studies have been conducted on various aspects of 
children living in intact, one-parent, and remarried families but findings 
from previous research studies are generally inconsistent and inconclusive 
(Camara, Baker, & Dayton, 1980; Goetting, 1983). According to Mensink 
(1987), there are three explanations for inconclusive findings in the re- 
search literature on children’s postdivorce functioning: (a) clinical and non- 
clinical samples could show different results; (b) intervening variables (e.g., 
IQ and SES) may differentially influence different studies; and (c) different 
results could be found depending on the type of measurements used (e.g., 
internal child perceptions or external parent and teacher perceptions). 
Amato and Ochiltree (1987) compiled 10 methodological limitations of pre- 
vious research studies on children’s postdivorce functioning. The present 
study was designed to meet seven of the limitations: 


1. the sample is not clinical but rather it was selected from the general 
population, 

2. children who lost a parent from death were excluded from the sample, 

3. intact two-parent families were distinguished from remarried step- 
families, 

4. the social class variable was partially controlled in that children were 
selected from schools in middle-class neighborhoods, 

5. two age levels were analyzed by selecting students from Grade 6 and 
Grade 9, 

6. child strengths were assessed by using the Health Resources Inventory, 
and 

7. multiple child outcome measures were used (cognitive, affective, be- 
havioral). 


The impact of family form on children’s functioning is not yet fully 
known due to equivocal findings from existing studies on this topic. Accord- 
ingly, the purpose of the present study was to determine whether family 
form is an important variable to consider with respect to the functioning of 
early adolescent children. 

Although research on children’s functioning in various family forms has 
often been inconsistent, there have been a few areas of progress during the 
last 10 years. Two key factors appear to enhance children’s functioning in 
any family form: (a) parental harmony (Ellison, 1983; Jacobson, 1978a; 
Kurdek, Blisk, & Siesky, 1981; Nelson, 1981; Rosen, 1979) and (b) positive 
interactions between children and their custodial and noncustodial parents 
(Hetherington, 1979; Jacobson, 1978b; Kurdek & Siesky, 1980; Pett, 1982). 
Various researchers (Emery, 1982; Goetting, 1983; Hetherington, 1979; 
Longfellow, 1979; Marotz-Baden, Adams, Bueche, Munro, & Munro, 1979; 
Rubin & Price, 1979; Schlesinger, 1977) conclude that children may be less 
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affected by family form than by the ongoing relationships between and 
among parents and children (family process variables). Accordingly, 
children’s perceptions of family processes were included in the present 
study in addition to the traditional cognitive, affective, and behavioral 
measures. 


Method 
Subjects 


The population from which the sample was selected consisted of Grade 6 
and Grade 9 children as well as their teachers and parents from middle 
socioeconomic status neighborhoods in Edmonton, Alberta, Canada. Schools 
were selected on the following basis: (a) each school was located in the Ed- 
monton Metropolitan Census Tract, (b) each school was located in the mid- 
dle socioeconomic status neighborhood based on the $26,000 to $35,000 
range of average family income, and (c) principals had to give permission 
for the participation of their schools in the study. In total 18 schools were 
contacted and principals of 13 schools agreed to participate. The re- 
searchers requested permission from the parents of all Grade 6 and Grade 9 
students in the 13 schools (V=950). Of the original 950 parents, 214 (23%) 
agreed to participate. Students were classified into the three family forms 
(intact—lives with two biological or adoptive parents; one-parent—lives 
with one biological parent; remarried—lives with one biological and one 
stepparent) on the basis of information provided by their parents. Two stu- 
dents were dropped from the analysis because they did not live in an intact, 
one-parent, or remarried family. 

The final sample included 136 students from intact families, 42 students 
from one-parent families, and 34 students from remarried families. The 
number of subjects in each family form, grade, and gender is presented in 
Table 1. The sample was quite balanced regarding grade and gender except 
that there were considerably more male than female students from Grade 9 
one-parent families in the sample. Student scores on the Canadian Cogni- 
tive Abilities Test (Thorndike, Hagen, & Wright, 1982) and the Lorge- 
Thorndike Intelligence Test (Lorge, Thorndike, Hagen, & Wright, 1967) 
were collected to determine if the children differed on IQ (group) across 
family form. No significant differences were found among the three groups 


Table 1 
Number of Subjects in Each Family Form, Grade, and Gender 


Grade 6 Grade 9 
Family Form Male Female Male Female Total 
tee a Re oD ee a ee Ss eee enener ee 
Intact 30 39 30 37 136 
One-Parent 10 Hf iv, 4 42 
Remarried 5 4 12 13 34 
Totals 45 54 59 54 212 
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(intact, one-parent, remarried). As a result, students from the three family 
forms were considered similar with respect to IQ for the data analyses. 


Instruments 


Le 
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Culture-Free Self Esteem Inventory (SEI), Form B (Battle, 1981). The 
SEI contains 30 items, five of which comprise the lie scale. The instru- 
ment without the lie scale contains 25 items intended to measure an 
individual’s perception in four areas: general self, social, school, and 
parents. The items are divided in two groups: those which indicate high 
self-esteem and those which indicate low self-esteem. The individual 
checks each item either yes or no. According to Battle, the instrument 
has content and concurrent validity and estimates of test-retest 
reliability range from .79 to .92 (Battle, 1981). 


. Family Assessment Measure (FAM) (Skinner, Steinhauer, & Santa-Bar- 


bara, 1983). The FAM is a self-report instrument that provides quantita- 
tive indices of family strengths and weaknesses. Family members are 
asked to indicate (strongly agree to strongly disagree) how well each 
item describes their family. The instrument is based on a process model 
of family functioning that integrates different approaches to family 
therapy and research. The FAM has three scales: General, Dyadic 
Relationship, Self. Only the General Scale was used for the present 
study due to time restrictions. The General Scale (50 items, nine subs- 
cales) focuses on the level of health/pathology in the family from a sys- 
tems perspective and provides an overall level of family functioning 
using the seven family process measures (subscales) and two response 
style subscales (Social Desirability and Denial). The seven measures 
relating to the process model are (a) task accomplishment—the achieve- 
ment of a variety of basic, developmental, and crisis tasks to reach 
biological, psychological, and social goals of the family; (b) role perfor- 
mance—the differentiation and performance of various roles which are 
prescribed and repetitive behaviors including a set of reciprocal ac- 
tivities with other family members; (c) communication—the exchange of 
information and achievement of mutual understanding among family 
members; (d) affective expression—the content, intensity, and timing of 
feelings among family members; (e) affective involvement—the degree 
and quality of family members’ interest in one another; (f) control—the 
process by which family members influence each other to sustain ongo- 
ing functioning or to allow the family to adapt its functioning in 
response to changing task demands; and (g) values and norms—the cul- 
tural and family background against which all basic processes must be 
considered. The seven scales comprise 35 items in total which were used 
in the present study. Estimates of internal consistency for the General 
Scale are .93 for adults and .94 for children while validity studies are 
still in progress (Skinner et al., 1983). 


. Family Satisfaction Scale (Olson & Wilson, 1982). The Family Satisfac- 


tion Scale is a self-report instrument which assesses family satisfaction 
on the dimensions and subscales of family cohesion (emotional bonding) 
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and family adaptability (ability to change) of the Circumplex Model of 
family functioning (Olson, Russell, & Sprenkle, 1983). The Family Satis- 
faction Scale is a 14-item instrument on which subjects respond to each 
item about their family on a five-point Likert scale: 1=dissatisfied, 
2=somewhat dissatisfied, 3= generally satisfied, 4=very satisfied, 5=ex- 
tremely satisfied. Eight of the items load on family cohesion and six 
items load on family adaptability. An estimate of internal consistency of 
the total instrument is .92 and test-retest reliability is .75 (Olson & Wil- 
son, 1982). 

4. Health Resources Inventory (HRI, Gesten, 1976). The HRI measures 
perceptions of competence-related behavior of children. For the present 
study, four of the 54 items were omitted and six items were slightly 
modified to adjust the HRI for teacher and parent completion. The 
revised inventory contains 50 items which are assigned a rating of 1 to 5 
(1=not at all, 2=a little, 3=moderately well, 4=well, 5=very well) ac- 
cording to how well the behavior describes the child. The HRI contains 
five subscales: (a) good student—items related to effective learning; (b) 
gutsy—items reflecting adaptive assertiveness or ego strength; (c) peer 
sociability—items reflecting effective interpersonal functioning; (d) 
rules—items reflecting the child’s ability to function within the con- 
straints of the environment; and (e) frustration tolerance—items reflect- 
ing the child’s ability to cope with failure and other social pressures. 
Although reliability and validity estimates are not presented by Gesten, 
Sattler (1982) suggests that reliability and validity estimates of the in- 
strument are satisfactory. 

5. School Report Grades. The academic report includes grades in language 
arts, mathematics, social studies, and science for all students in the dis- 
trict. Scores on a 100-point scale were converted for the study to a 1 
(low) to 5 (high) rating on the basis of the Edmonton Public Schools 
standard grid—O0 to 39=1, 40 to 49=2, 50 to 64=3, 65 to 79=4, 80 to 
100=5. 


Procedure 

A consent form, the Health Resources Inventory, and a background infor- 
mation form were mailed to parents. Next, the Culture-Free Self Esteem 
Inventory, the Family Assessment Measure, and the Family Satisfaction 
Scale were administered to the participating students in their classrooms. 
At the same time, the Health Resources Inventories were given to the 
homeroom teachers for their completion. Also, group ability test scores and 
school report grades were collected from student records in each school. 


Results 
Investigation of Family Form—Analysis of Variance (ANOVA) Results 


A three-way analysis of variance model was used for the initial analysis of 
each child functioning variable—3 (family form: intact, one-parent, remar- 
ried) by 2 (grade: 6, 9) by 2 (gender: male, female). The SPSSx Program 
was used to compute the ANOVA (SPSS, Inc., 1986). The F' values of each 
interaction and main effect are displayed in Table 2. Multiple comparisons 
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Table 2 
ANOVA Summary of F Values 
Form 
by 
Form Form Grade Grade 
by by by by 

Variables Form Grade Gender Grade Gender Gender Gender 
Self-Esteem 

Inventory A77***) -3.05 2.57 0.49 0.10 0.45 2.17 
Family Assess- 

ment Measure 0.37 0.80 0:31- 3.26* 1.01 0.68 1.41 
Family Satisfac- 

tion Scale 1:22 3.56 4.50* 1.15 0.48 1.90 0.47 
Health Resources 

Inv Teacher 9.59**** 7 .0,83 4.86* 1.91 0.52 0.04 0.84 
Health Resources 

Inv Parent TAO sae 0.33 7.94** 0.00 0.32 0.18 0.23 
School Report 

Grades SHY oda) 0.75 0.74 0.91 1.45 1.45 0.10 
*p <.05 
**y 04 
call oc-4. (9,9) 


between the means of each level were calculated using the Tukey HSD 
procedure (Kirk, 1968). 

None of the three-way interaction effects was significant. The only sig- 
nificant two-way interaction effect was the family form by grade effect on 
the Family Assessment Measure (F=3.26, p<.05). The Tukey HSD test 
showed that Grade 6 students from remarried families had a higher mean 
score on the Family Assessment Measure than the Grade 9 students from 
remarried families (mean=107.9 and 94.5, respectively). Only one interac- 
tion was significant and, as a result, emphasis will be placed on the results 
of the main effects. 

The following main effects for family form were found significant: Self- 
Esteem Inventory (F=4.77, p<.01), Health Resources Inventory Teacher 
(F=9.59, p<.001), Health Resources Inventory Parent (F=7.40, p<.01), 
and School Report Grades (F=8.37, p<.001). 

The results of the comparisons of means are presented according to the 
groups compared: (a) intact and one-parent, (b) intact and remarried, and 
(c) remarried and one-parent. Children from intact families received a 
higher mean score on teacher ratings and school grades than children from 
one-parent families. In addition, children from intact families received a 
higher mean score on self-esteem, parent ratings, and school grades than 
children from remarried families. Finally, children from remarried families 
received a higher mean score on teacher ratings than children from one- 
parent families. The means and standard deviations of the child functioning 
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variables across family form are presented in Table 3. With the exception of 
the mean difference on self-esteem between children from intact and 
remarried families, the major differences regarding family form between 
the three groups are external to the children (teacher ratings, parent 
ratings, and school grades) in favor of the intact family group. 

There were no main effects for grade. The following main effects for 
gender were found significant: males received a higher mean score on fami- 
ly satisfaction than females (F=4.50, p<.05), females received a higher 
mean score on Health Resources Inventory Teacher (teacher rating) than 
males (F=4.86, p<.05), and females received a higher mean score on 
Health Resources Inventory Parent (parent ratings) than males (F=7.94, 
p<.01). The means and standard deviations of the child functioning vari- 
ables across gender are presented in Table 4. On the basis of these results, it 
would seem that external observers such as teachers and parents rate 
females higher than males. 

In summary, three major statements can be made on the basis of the 
ANOVA results. First, children from intact families received higher mean 
scores than children from one-parent families on two variables (teacher 
ratings and school grades), and children from intact families received 
higher mean scores than children from remarried families on three vari- 
ables (self-esteem, parent ratings, and school grades). This means that 
family form is an important variable regarding the perceived functioning of 
children. Such a conclusion is primarily true when child functioning is con- 
sidered from an external perspective (teachers and parents). Second, there 
were no significant differences between children from intact, one-parent, 
and remarried families on the variables tapping children’s perceptions of 


Table 3 
Means and Standard Deviations of Child 
Functioning Variables Across Family Form 


Intact One-Parent Remarried 
Std Std Std 
Variables Mean Dev N Mean Dev N Mean Dev N 
Self-Esteem 
Inventory 20. 1pedi26.- 8185 19,2 Be 3.88 42 18.4 4.41 33 
Family Assess- 
ment Measure 100. feeta5o o125 100! Zamna OOMmEGO 98.2 15.56 29 
Family Satisfac- 
tion Scale SLSR Re PRES Bt) 47.7 7.91. 41 45.3. 10./4euiae 
Health Resources 
Inv Teacher 191.8 29.12 124 162.5 45.66 39 187.3 635,00, 6 27 
Health Resources 
Inv Parent 193.7 28.86 134 178.7 31.24 41 175.3 26.63 33 


School Report 
Grades 164. 3:33. 134 T.7 3.88 42 142 .3.98 33 
She ivOne 16 Wh SO Cet seh os eS ee 
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Table 4 
Means and Standard Deviations of Child 
Functioning Variables Across Gender 


Male Female 

Std Std 
Variables Mean Dev N Mean Dev N 
Self-Esteem Inventory 20.0 3.81 102 19.3 3.44 108 
Family Assessment Measure 100.7. 13.51 92 99.7 14.24 97 
Family Satisfaction Scale 49.1 9.24 103 46.6 9.78 105 
Health Resources Inv Teacher 177.9 39.05 92 192.0 30.97 98 
Health Resources Inv Parent 181-55 33,12 102 194.0 24.88 106 


School Report Grades 14.9 3.71 101 15.6 3.65 108 


their families—family process and family satisfaction. Children’s families 
provide similar interactions regardless of family form. Third, the results did 
not vary significantly on either age or gender. 


Investigation of Family Form—Exploratory Data Analysis (EDA) Results 


The results of the six measures of children’s functioning (Self Esteem, 
Family Assessment Measure, Family Satisfaction Scale, Health Resources 
Inventory Teacher, Health Resources Inventory Parent, and School Report 
Grades) were compared across family form (intact, one-parent, remarried) 
using box and whisker graphs or boxplots (Tukey, 1977). These were con- 
structed for the six variables and are presented in Graphs 1 to 6. 

The boxplots provide clear pictorial representations of the central ten- 
dency (median) and spread of the six variables under study. The box con- 
sists of scores between the first and third quartiles or the middle 50% of the 
distribution including a line which divides the box into two separate parts 
and represents the median of the data array. The whisker extends from the 
box to the extreme values (highest, lowest, adjacent). If values lie outside 
upper or lower fences (rule of thumb is 1.5 times the distance between the 
upper and lower quartiles) then the whisker extends to the adjacent value 
and the outlier scores are marked by dots. The outliers are indicated by C 
(ID#) on Graphs 1 to 6. 

The aim of the exploratory analysis was to compare the shape and 
spread of the distribution across groups as well as to note exceptional 
points. It was thought that while the trends across groups could be com- 
pared by the analysis of variance procedure, individual anomalies could go 
undetected and yet be indicative of low frequency effects. 

The following observations can be made about the boxplots. First, the 
three family form groups are not vastly dissimilar with respect to the spread 
of the distributions of the six variables. The majority of the children from 
the sampled intact, one-parent, and remarried families are at about the 
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Intact One-Parent Remarried 
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Graph 1; Boxplot of family satisfaction 
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Intact One-Parent Remarried 
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Graph 2: Boxplot of self-esteem 
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Graph 3: Boxplot of family assessment measure 
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Graph 4: Boxplot of health resources inventory teacher 
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Intact One-Parent Remarried 
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Graph 5; Boxplot of health resources inventory parent 
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Graph 6: Boxplot of school report grades 
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same level on the six variables. Second, with only one exception (Child #111 
on FAM and FS—Graphs 2 and 3), there are no children who are outliers on 
two or more variables. That is, the samples do not contain extreme low or 
high values of an individual over many variables. Therefore, the sample 
does not contain children with severe problems in many areas (achieve- 
ment, behavior ratings, self-esteem, etc.). This finding provides confirma- 
tion of the nonclinical nature of the sample. Third, with the exception of the 
remarried group on school grades (Graph 1), the whiskers (extent of spread) 
are similar across family form. This finding provides further confirmation 
of the similarity of the sample across family form. Finally, it would appear 
that there is greater similarity among the medians and distributions of the 
self-esteem (Graph 1), child perception of family processes (Graph 2), and 
child satisfaction (Graph 3) variables across family form than for teacher 
perceptions (Graph 4), parent perceptions (Graph 5), and school grades 
(Graph 6). 

The differences observed on Graphs 4, 5, and 6 are as follows: (a) the 
boxplots of teacher ratings, parent ratings, and school grades show higher 
scores for children from intact families than for children from one-parent 
families; (b) the boxplots of parent ratings and school grades show higher 
scores for children from intact families than for children from remarried 
families; and (c) the boxplot of teacher ratings shows higher scores for 
children from remarried families than for children from one-parent 
families. These results are generally consistent with the ANOVA results ex- 
cept differences on self-esteem are not apparent in examining the boxplot 
(Graph 1) and the boxplot of the intact group on parent ratings (Graph 5) 
shows higher scores than the boxplot of the one-parent group. 

The more recent exploratory procedures provide additional information 
to the traditional ANOVA results because EDA results are less affected by 
extreme scores and because entire distributions of variables can be com- 
pared which, in turn, provides a more complete picture of the results. Also, 
EDA results are more sensitive to exceptional points. In short, the results of 
the EDA indicated that the children from each family group belong to the 
same population of children—boxes and extensions of whiskers were not 
vastly dissimilar. These results suggest that it would be difficult for a 
teacher to distinguish between children living in various family forms 
without prior knowledge. In addition, the EDA results further confirm the 
finding of the ANOVA procedure that children from various family forms do 
not perceive themselves as being different (internal focus) but that teachers 
and parents perceive children from various family forms as being different 
(external focus), in favor of children from intact two-parent families. 


Relationships of Variables—Pearson Correlations 


Correlations calculated among the perceptions of child functioning variables 
are presented in Table 5. An outstanding feature of the Pearson correla- 
tions is that there are two clusters of variables. The child perception vari- 
ables (self-esteem or perception of self, perception of family process, family 
satisfaction) are highly correlated with each other and the external percep- 
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Table 5 
Pearson Correlations Among the Dependent Variables 


Self Fam Fam . HRI HRI 
Esteem Assess Satis Teach Parent 
(N) (N) (N) (N) (N) 
Family Assessment Measure sj aa 
(188) 
Family Satisfaction Scale Jenne Ose 
(207) (187) 
Health Resources Inv Teacher WPM cake" Bh oh 12 
(189) (170) (187) 
Health Resources Inv Parent 5-5 wae “Ne foun 
(206) (186) (205) (188) 
School Report Grades oe baie tO Asi Petehy .40*** 
(207) (187) (205) (188) (205) 
*p <.05 
***p <.001 


tions of child variables (teacher rating, parent rating, school grades) are 
highly correlated with each other. The other correlations are below .30. 
Therefore, the Pearson correlations show two major clusters—one internal 
and one external to children. The implications of this distinction are dis- 
cussed in the next section. 


Discussion 

The perceptions of early adolescent children’s functioning selected for the 
present study clumped into two factors—a weak external (outsider) factor 
and a stronger internal (insider) factor. The factors can be considered 
relevant in understanding the results of the study because the findings of 
the study were different for each factor. In general, children from intact 
families received more favorable external ratings than children from one- 
parent and remarried families. Accordingly, on the basis of these variables 
of children’s functioning, one would assert that children from a nonclinical, 
middle-class population differ across family form in favor of the intact, two- 
parent family form. However, on the basis of internal (child) perceptions of 
themselves and their families, one would assert that family form does not 
differentiate or distinguish children from intact, one-parent, and remarried 
families, 

One interpretation of these findings is that teachers and parents expect 
children from one-parent and remarried families to show problems. 
Santrock and Tracy (1978) found that the behavior on a videotape record- 
ing of the same child was perceived more positively by teachers when they 
were told that the child lived in an intact family instead of in a divorced 
family. These results may indicate that children are less influenced by fami- 
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ly form than their teachers or parents. That is, it could be that the external 
perceptions of children who have experienced family transitions are con- 
gruent with a negative bias toward divorce and alternative family forms as 
opposed to a true indication of children’s functioning. Moreover, the exter- 
nal perceptions of children’s functioning in the present study might be a 
measure of adult experiences and perceptions of family transitions. 

Of course, an alternative explanation of the findings is that the teacher 
and parent perceptions are accurate and children’s perceptions are an inac- 
curate map of reality. This explanation is suspect for two reasons. First, 
there was no reason or possible gain for children to misrepresent their per- 
ceptions. Second, a person’s perceptions of relationships or of an event are, 
in fact, reality for that individual. In discussing the ideas of Bateson and 
Maturana, Andersen (1987) states that “both agree that it is the observer 
who generates the distinctions we call ‘reality.’ One’s picture of or 
knowledge about the world will be the basis for one’s attitude to it”(p. 416). 
Perhaps the sampled children’s perceptions (internal perspective) are repre- 
sentative of a child’s map of reality and the sampled parents and teachers’ 
perceptions (external perspective) are representative of an adult map of 
reality. 

The findings of the present study suggest that, despite family transition, 
children living in one-parent and remarried families indicate that they are 
as satisfied with their families as children living in intact families. 
Children’s perceptions of themselves and of themselves in their families 
provide insight into their functioning. Internal perceptions or the view one 
has of oneself and of the interactions between oneself and those in one’s ex- 
ternal world (family, school, etc.) could be considered more indicative of 
children’s functioning than external perceptions. The way a person per- 
ceives her or his peculiar experiences is a core conceptual issue of the family 
sociological theory of symbolic interactionism. Specifically, the key con- 
struct of definition of the situation is relevant to the findings of the present 
study. Stryker and Statham (1985) state that “the underlying premise of 
symbolic interactionism is that the subjective aspects of experience must be 
examined because the meanings people assign to things ultimately organize 
their behavior” (p. 320). Persons create the meanings to which they respond 
through the process of social interaction. According to Stryker and 
Statham, “that meaning is derived from an interpersonal process implies 
that reality is defined through that process rather than independent of it” 
(p. 321). The findings of the present study suggest that the family process 
or interactional aspects which contribute to children’s family definition are 
independent of family form. That is, children did not differ with respect to 
family perceptions across family form. 

If children do not differ significantly across family form, then how does 
one account for the negative findings from many previous studies regarding 
the effect of divorce on the functioning of children (Marotz-Baden et al., 
1979)? First, as is pointed out previously, external perceptions show child 
problems, and many studies have used external measures of child behavior. 
Second, many studies have been conducted on clinical rather than norma- 
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tive samples. Third, many researchers conducting studies in the area of 
children’s functioning and family transitions over the last 10 years conclude 
that family form may be less important than family process (relationship 
patterns) to children’s functioning (Amato & Ochiltree, 1987; Emery, 1982; 
Hess & Camara, 1979; Hetherington, 1979; Longfellow, 1979, & Marotz- 
Baden et al., 1979). In discussing the results of their study, Amato and 
Ochiltree (1987) state the following: 


Although significant differences were obtained between family types, the family type 
variable only accounted for small proportions of the variance in each measure.... 
Therefore, in future work we intend to examine family process measures to see what 
the intervening mechanisms might be which account for the differences in family types 
noted earlier. It is anticipated that the levels of family conflict and the quality of 
parent-child relationships will be important variables in this analysis. (p. 94) 


According to the findings reported in the present study, family process, at 
least from the perspective of children, can be of similar quality regardless of 
family form. It seems that family process in postdivorce families is certainly 
worth serious investigation. 

There is a great deal of optimism involved in the hypothesis that family 
process is more important to the functioning of children than family form. 
To explain, families have been and will be undergoing changes in form. 
That families change through divorce and remarriage will likely not change. 
On the other hand, processes in families do change and are amenable to 
changes. Consequently, from this perspective, children’s functioning is more 
dependent on a condition subject to change rather than something un- 
amenable to change; herein lies the optimism. 

A final implication of the present study is a practical matter directed to 
those who work with children and families experiencing transition. If 
children’s definition of the situation organizes their experience and if their 
definition of the situation is dependent on social interactions, then it is cru- 
cial for therapists and mental health workers to be open to the positive 
aspects of family transition. Wilcoxon (1985) states the following on this 
topic: 

A counselor’s awareness of healthy family characteristics could be quite valuable in 

providing assurance that healthy family processes are not completely eroded, but 

rather are undergoing adjustment and change. In fact, it is quite possible that these 


very characteristics have brought the family to its current stage of readiness for entry 
into a new phase of growth and development. (p. 498) 


Mental health workers, therapists, and educators can provide hope for 
children and families by offering supportive affirmation during transitions. 
The supportive affirmation will, in turn, help to enhance a positive defini- 
tion of family transition, and so forth. 

In conclusion, the results of the present study point to the relevance of 
considering internal variables in addition to external variables regarding 
the functioning of children from various family forms. It was suggested that 
symbolic interactionism theory might help in understanding the relation- 
ship between internal perceptions and the experiences of children in terms 
of their functioning. The findings indicated that family form might not be 
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the most important variable to consider in understanding children’s 
functioning. Finally, the results point to the need for further research on 
the relevance of family process to the functioning of children from any or all 
family forms. 
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This study examined the effects of gender, ethnicity, and achievement on ratings of ap- 
propriateness of type of high school program (academic, commercial, vocational) for grade 
9 students. The subjects—third- and fourth-year undergraduate Education students—were 
each asked to evaluate a case folder in which gender (female, male, not stated), ethnicity 
(Native, non-Native), and achievement (higher average, lower average) were systematically 
varied. Subjects provided ratings of appropriateness of type of program on five-point 
Likert-type scales. Recommendations for programs were influenced by student attributes. 
Academic programs were rated more appropriate for Native than non-Native students; 
commercial programs were rated more appropriate for young women; and academic pro- 
grams were rated more appropriate for higher-achieving than lower-achieving students. 
Implications for school counseling and future research are discussed. 


Every high school student is faced with decisions about program selection 
and occupations and careers to pursue after leaving school. Guidance coun- 
seling is a feature of high schools designed to assist with these decisions, 
although many Canadian and American schools lack the resources to enable 
every student to receive program and career advice from a counselor. In a 
nationally representative sample of American students, approximately 50% 
reported no contact with a guidance counselor when making these important 
decisions (Lee & Ekstrom, 1987). Students with lower academic scores and 
those from lower socioeconomic status homes were even less likely to receive 


Elisabeth Stewart is a recent graduate of the educational psychology program at the 
University of Regina. 

Nancy Hutchinson is an assistant professor in educational psychology at Queen’s 
University whose research interests include influences on learning and development in 


adolescents. , 
Peter Hemingway is a professor of educational psychology at the University of Regina 


conducting research on teacher decision making. ' 
Frederick Bessai is a professor of educational psychology at the University of Regina 


interested in adolescent learning. 


255 


E.M. Stewart, N.L. Hutchinson, P. Hemingway, F. Bessat 


counseling. However, teachers have considerable influence on students, and 
every high school student has contact with several teachers every day. 
Farmer (1985) showed that the.influence of personal factors such as self-es- 
teem and attributions on the aspirations of adolescents was mediated by 
environmental factors. There were gender differences, with young men more 
influenced by parent support and young women more influenced by teacher 
support in their environment. 

If teachers are one of the major sources of advice and support for high 
school students, then it is important to know the quality and appropriateness 
of the advice given by teachers. This is particularly true for young women in 
their high school years who are more influenced by their teachers. 

This study addressed the question of what kind of program advice 
Canadian preservice teachers would give to students as a function of specific 
student characteristics. The rationale for the study is based on two phe- 
nomena: occupational stratification and differential success in schools. The 
lives of a disproportionate number of Canadian Natives are characterized by 
casual or seasonal labor and frequent periods of unemployment (Department 
of Indian and Northern Affairs, 1980), and women are clustered in clerical 
and service occupations (Devereaux & Reichnitzer, 1980; Statistics Canada, 
1980). Also, Natives do not fare well in our schools (Ellis, Pinacie, Turner, & 
Swiftwolfe, 1978; Department of Indian and Northern Affairs, 1980). Some 
of the causes of occupational stratification may be traced to educational 
decisions made in the adolescent years (Farmer, 1987; Kolbenschlag, 1979), 
and it is reasonable to suppose that success in school is related to decisions 
about the choice of school program. Teachers’ advice, as we have noted, can 
be expected to influence these decisions. However, we do not know specifical- 
ly what factors or groups of factors determine the kind of program advice 
given by teachers to high school students. 

Studies demonstrate the relationship between student characteristics and 
teachers’ judgments. For instance, teachers’ expectations are influenced by 
specific student characteristics (Brophy, 1981; Brophy & Good, 1974). 
Recent studies have demonstrated that the gender of students influences the 
program expectations held for them by school personnel (Ott, Carmichael, 
Gallup, Lewis, Russ, & Veres, 1980). Ethnicity of students has been shown to 
have significant effects on expectations held by preservice and inservice 
teachers (Clifton, 1981, 1982, 1986; Finn, 1972; Herbert, Hemingway, & 
Hutchinson, 1984; Hutchinson & Hemingway, 1984). A third variable— 
achievement—has also been shown to be a significant influence in forming 
teachers’ expectations of students (Benz, Pfeiffer, & Newman, 1984; Finn, 
1972; Rolison & Medway, 1985). Gender, ethnicity, and achievement have 
each been shown to influence teachers’ expectations for students. However, 
it is not enough to know that teachers’ expectations are influenced by student 
characteristics; it is essential to know if these expectations translate into 
different kinds of program advice being given to students with different 
attributes. 

This study investigated whether program advice given by teachers was 
influenced by student attributes of gender, ethnicity, and achievement, singly 
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and in combination. Program advice given by teachers may have a mediating 
influence between student attributes and occupational aspirations and ac- 
complishments. 


Method 
Description of the Sample 


A total of 125 student volunteers from second-, third-, and fourth-year 
Education classes at a university in western Canada participated in this 
study. The sample was 106 women and 19 men; the age of the subjects 
ranged from 19 to 51 years with a mean age of 24.41 years (median=21.38) 
and a standard deviation of 6.88 years. Teaching experience of subjects 
ranged from 0 to 25 years, with a mean of 1.23 years (median=0.18) and a 
standard deviation of 3.53 years. Twenty-four of the subjects were in secon- 
dary education, 97 in elementary, and four did not indicate type of program. 
The number of classes previously taken in either educational psychology or 
guidance and counseling ranged from 0 to 8, with a mean of 1.63 and a 
standard deviation of 1.45. 

In a standard instruction sheet, the subjects were informed that participa- 
tion in the study was voluntary, that their responses would be anonymous, 
and that their participation would in no way affect their course grade. They 
were also informed that the results of the study would be made available to 
them after data analysis. 


Procedure 


Subjects were assigned the task in group settings. They were given verbal 
and written instructions to imagine that they were grade 9 classroom teach- 
ers. One of their responsibilities at the end of the year was to make specific 
recommendations concerning their students. Each subject was asked to ex- 
amine the contents of a file folder, and, by using the information therein, to 
respond to a questionnaire. On the questionnaire they were asked to rate the 
appropriateness of each of three possible education programs: academic, 
commercial and business, and vocational and technical. For each of these 
questions, the subjects’ responses were recorded using five-point Likert-type 
scales from very appropriate to very inappropriate. Subjects were given def- 
initions of the three educational programs. They were also asked to provide 
demographic information about themselves: gender, age, teaching experi- 
ence, and classes taken in educational psychology, guidance, or counseling. 

Each subject reviewed one case folder and responded to the questionnaire. 
There were 12 experimental conditions to which subjects were randomly 
assigned. The conditions represented all possible combinations of the inde- 
pendent variables in a 3x2x2 complete factorial design, that is, gender (male, 
female, not stated) x ethnicity (Native, non-Native) x achievement (higher 
average, lower average). Subjects did not know that others were responding 
to case folders containing different information. 


Case Folders 


In total 120 case folders were prepared (10 identical folders for each of the 
experimental conditions). The age and grade level of the individual repre- 
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sented in the case folder were held constant throughout (15 years, seven 
months; grade 9), while the independent variables were systematically varied 
to reflect the appropriate conditions. 

The gender of the student represented in the case folder was given by 
direct listing; for the four “gender not stated” experimental conditions, all 
references to the gender identity of the student were blacked out. Ethnic 
identification (Native, non-Native) was provided by varying the student’s 
last name (Stonechild, Johnston) and listing the language of the home (Cree, 
English). Stonechild is a common name for Cree Natives in western Canada, 
and Johnston is a common name for non-Natives. Although many Cree 
children speak English at home, the language of the home was stated in the 
case folder as Cree to add to the readers’ impression that a student named 
Stonechild was definitely a Cree Native. 

The level of achievement (higher average, lower average) was given by 
intelligence test data (WISC-R profile) and teacher-assigned final marks for 
grade 9. The higher average student was identified as one standard deviation 
above the mean for both IQ (full-scale IQ=115) and class average (75%). The 
lower average student was identified as having both a full-scale IQ of 85 (-1 
SD) and a class average of 55% (—1 SD). On the WISC-R profile, care was 
taken to ensure that there were no differences in the two profiles of subscale 
scores (Sattler, 1974). Similarly, in assigning subject marks, care was taken 
to ensure that the higher and lower average students had parallel grade 
profiles. 


Results 

In this study, subjects were asked to choose the level of appropriateness for 
three types of school programs. The independent variables manipulated in 
the case folders were gender (female, male, gender not stated), ethnicity 
(Native, non-Native), and achievement (higher average, lower average). The 
results were analyzed using a four-way analysis of variance with the fourth 
variable (program) having correlated levels. The significance level for statis- 
tical testing was set at .05. 

The means and standard deviations of the subjects’ responses on the 
five-point Likert-type scale are reported in Table 1 (where 1=very appro- 
priate and 5=very inappropriate). Strong ratings of appropriateness were 
given to the academic program for cases describing Native students and 
non-Native students with higher achievement: Native males with higher 
achievement (1.30); Native students with gender not stated and higher 
achievement (1.70); Native females with higher achievement (1.80); and 
non-Native students with gender not stated and higher achievement (1.91). 
A strong rating of appropriateness was also given to the vocational program 
for non-Native males with lower achievement (1.60). These means were all 
less than 2 on a scale where 1 represented very appropriate. Table 2 presents 
a summary of the analysis of variance for programs. Two main effects were 
Significant, as were three interaction effects. The interactions are discussed 
first as they involved the interaction of program with the variables of inter- 
est: gender, ethnicity, and achievement. 
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Table 1 
Means and Standard Deviations for Ratings of Appropriateness of Programs 
as a Function of Gender, Ethnicity, and Achievement 


Student Information Program 


Gender Ethnicity Achievement N Academic Business - Vocational 
M2.°+ SDigee M3) (SDaqe Me esp 


Female Native Higher 10 1.80 084 240 084 3.30 0.84 
Lower 1 S18 0.598 1) 2.50.08 0.650 ee soem Bo 
Non-Native Higher 10 2.10 084 2.70 084 3.20 0.84 
Lower 10 3.30 084 230 084 2.20 0.84 
Male Native Higher 10 1.30 084 380 084 3.80 0.84 
Lower 10 3.70 084 340 084 260 0.84 
Non-Native Higher 11 150 084 300 084 3.00 0.84 
Lower 10 3.70 084 2.70 084 1.60 0.84 

Not 
Stated Native Higher 10 1.70 084 3.10 084 300 0.84 
Lower 11 3.35 082 3:09 082° 2.27 ~ 0.62 
Non-Native Higher 1 T91 ©6082) 2:27. US82 2.73 4082 
Lower 11 3.80 084 290 084 2.10 0.84 


&Means represent scores ranging from 1 (very appropriate) to 5 (very inappropriate). 


Table 2 — 
Analysis of Variance for Ratings of Appropriateness of Programs 


Source of Variance® DF Mean Square F 
Gender 2 1.50 1.36 
Eth 1 4.52 4.08* 
Gender X Eth 2 2.28 2.06 
Ach 1 6.51 5.89* 
Gender X Ach 2 1.41 1.27 
Eth X Ach 1 04 04 
Gender X Eth X Ach 2 61 tele 
Error 1 111 tod 

Program 2 1.64 2.07 
Gender X Program 4 2.66 3.36* 
Eth X Program a 3.72 4.70* 
Gensler X Eth X Program 4 54 68 
Ach X Program 2 62.07 78.45* 
Gender X Ach X Program 4 1.69 2.14 
Eth X Ach X Program 2 07 .09 
Gender X Eth X Ach X Program 4 .20 vas) 
Error 2 222 .79 

le eet i 2 SMS OS 
*p <.05 


&Code: Gender = gender of stimulus person; Eth = Ethnicity (Native/Non-Native); 
Ach = Achievement (higher/lower) 
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The gender by program interaction was significant F(4,222)=3.36, 
p<.05. Post hoc analysis using the Newman-Keuls procedure showed that 29 
of the 36 differences were greater than the critical values for significance. 
The means of the three gender groups (male, female, not stated) for each 
program (academic, business, and vocational) are presented in Table 3 and 
Figure 1. Examination of these data demonstrates that the business program 
was judged the most appropriate for women (with the lowest rating of the 
nine combinations where low numerical rating represents high judgment of 
appropriateness). Conversely, the business program was judged as least ap- 
propriate for men; it obtained the highest numerical rating of the gender by 
program combinations. Figure 1 indicates less striking differences in ratings 
of appropriateness of academic and vocational programs on the basis of 
gender. 

The second significant interaction was ethnicity by program 
F(4,222)=4.70, p<.05. A Newman-Keuls analysis showed that all 15 com- 
parisons of means yielded significant differences greater than the critical 
values. Table 4 and Figure 2 demonstrate that the academic program was 
seen as most appropriate for Native students, with the business and voca- 
tional programs judged less appropriate for Native students. Conversely, the 
academic program was perceived to be less appropriate for the non-Native 
students described in the case folders, and the business and vocational pro- 
grams were judged more appropriate for the non-Natives. 

Achievement by program was the third significant interaction, 
F(2,222)=78.45, p<.05. Post hoc analyses (Newman-Keuls) showed that 14 
of the 15 differences between means exceeded the critical values. The means 
appear in Table 5. This interaction between achievement and program is 
shown graphically in Figure 3. The academic program was judged most 
appropriate for higher-achieving students with the most extreme mean of the 
study (1.73 on a scale where 1 represents very appropriate). At the other 
extreme, lower-achieving students received a mean of 3.53 for appropriate- 
ness of academic programs, thus indicating that the preservice teachers 
judged academic programs inappropriate for lower average achievers in this 
study. Business was judged equally appropriate for both groups of achievers, 


Table 3 
Means for Ratings of Appropriateness of Programs 
for the Gender X Program Interaction 


Program 
Gender Academic Business Vocational 
Male 2.56 3.16 2.84 
Female 2.60 2.49 2.79 
Not Stated 2.74 2.84 2.53 


Note: Means represent scores ranging from 1 (very appropriate) to 5 (very inappropriate). The 
lower the mean, the stronger the rating of appropriateness. 
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Table 4 | 
Means for Ratings of Appropriateness of Programs 
for the Ethnicity X Program Interaction 


Program 
Ethnicity Academic Business Vocational 
Native 2.54 3.06 2.91 
Non-Native 2.73 2.60 2.43 


Note: Means represent scores ranging from 1 (very appropriate) to 5 (very inappropriate). The 
lower the mean, the stronger the rating of appropriateness. 


Table 5 
Means for Ratings of Appropriateness of Programs 
for the Achievement X Program Interaction 


Program 
Achievement Academic Business Vocational 
Higher 1.73 2.83 3.13 
Lower a Hees! 2.82 2.21 


Note: Means represent scores ranging from 1 (very appropriate) to 5 (very inappropriate). The 
lower the mean, the stronger the rating of appropriateness. 


3n00) 
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Figure 1: Gender x program interaction 
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Figure 2: Ethnicity x program interaction 
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Figure 3; Achievement x program interaction 
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and the data for vocational programs were opposite to the academic program 
judgments, though the differences were less extreme (Figure 3). 

The two significant main effects were for ethnicity and achievement 
(Table 1), with programs judged as more appropriate overall for Native and 
higher-achieving students. However, these main effects may be attributed to 
particular significant interactions that have already been discussed. 


Discussion 

In this study we report the effects of gender, ethnicity, and achievement on 
preservice teachers’ judgments of the appropriateness of three programs for 
students described in case folders. Limitations of the procedure and the 
subjects who participated should be noted. The need to manipulate orthog- 
onally and to control each of the independent variables in this study dictated 
the use of simulated case histories. While this experimental procedure is 
widely used, it does have drawbacks. Possibly preservice teachers would 
make different decisions when interacting with students in the classroom 
than when reading about fictitious students. Also, their recommendations 
might change over the time of their contact with the students (Dusek & 
Joseph, 1983). These questions will remain unanswered until a number of 
studies have been conducted in classrooms. 

Two important considerations concerning the subjects who participated 
in this study should be noted. First, most of the subjects had no previous 
full-time teaching experience; it would be unwise to generalize these results 
to experienced Canadian teachers without confirmation that experienced 
teachers provide similar responses. Second, the majority of these subjects 
were women. Because most Canadian high school teachers are men, this 
sample did not accurately reflect the characteristics of the population to 
which we wished to generalize. The subsample of men was too small to allow 
for separate analysis; consequently, the question of the influence of subject 
gender has not been addressed. Although the two preceding considerations 
caution against generalizing these results to other groups without further 
study, the findings are decisive and surprising enough to warrant both 
concern and future research. The studies of Smith and Hemingway (1987a, 
1987b, 1987c) have begun this process by using a sample of experienced 
secondary teachers that closely approximates the gender ratio of the popula- 
tion of high school teachers in western Canada. Their results corroborate the 
findings of the present study. 

The finding that preservice teachers’ judgments about the appropriate- 
ness of particular programs were influenced by gender, ethnicity, and 
achievement supports and extends previous research. The interaction of 
gender and program illustrates the continued existence of the traditional 
stereotypes found in other studies (Oregon Department of Education, 1981, 
Ott et al., 1980; Picou & Howard, 1976; Russell, 1978; Smith & Hemingway, 
1987a, 1987b, 1987c). Subjects in this study rated as most appropriate the 
commercial program for young women, while the same program was rated as 
least appropriate for men. Recommending commercial programs for young 
men or for those students whose gender was not stated was something these 
subjects were not generally prepared to do. They demonstrated in their 
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ratings that gender-roles are an important determinant of appropriateness- 
of-program recommendations. 

Preservice teachers in this study found the commercial program to be a 
more appropriate recommendation for young women. Young women are 
particularly influenced by their teachers’ recommendations (Farmer, 1985) 
although they think that their program choices are not influenced by teacher 
advice (Gaskell, 1987). They may be most limited by advice and least aware 
of the source of the limitations as external to them. Clerical occupations of 
low status and low pay commonly follow from commercial programs in high 
school (Menzies, 1981), and women frequently find themselves limited to 
these positions (Conway, Bourque, & Scott, 1987; Gaskell, 1987; Marini & 
Brinton, 1984; Reid, 1981). If the findings of the present study are supported 
by additional research, an important factor in promoting occupational strat- 
ification by gender may have been identified. 

Preservice teachers’ program recommendations were influenced by the 
ethnicity of the student described in the case file. For the Native students, the 
program seen as most appropriate was the academic program, followed by 
the vocational program, and business was seen as least appropriate for this 
ethnic group. Conversely, the academic program was seen as least appro- 
priate for the non-Native students, with the vocational program viewed as 
most appropriate for the non-Native students described in the case files. 
Native students tend to leave school before completing high school, and those 
who do finish a secondary program rarely complete the academic program 
(Vernon, 1984). Perhaps in light of the paucity of Native adolescents thriving 
in western Canadian high schools, the subjects chose to encourage these 
relatively successful Native students to pursue academic goals. The Native 
students described may have been seen as outstanding for this reference 
group. This might be interpreted as reverse discrimination because non-Na- 
tive students with the same sets of scores were viewed as more appropriate 
candidates for vocational and business programs than for academics. These 
data suggest that educators may advise Native students to enroll in academic 
studies; however, these data say nothing about educators’ willingness to 
create the classroom conditions necessary for Native students to succeed 
(Kleinfeld, 1975). Advising Native students to enroll in academic programs 
may need to be accompanied by research on variables that intervene between 
ethnicity and academic achievement (e.g., Clifton & Roberts, 1988). 

Program recommendations were influenced by achievement, with higher- 
achieving students seen as more appropriate for academic programs and 
lower-achieving students for vocational programs. Two combinations were 
deemed to be absolutely inappropriate, with a mean greater than three: 
higher-achieving students in vocational programs, and lower-achieving stu- 
dents in academic programs. Because achievement data used in this study 
represented lower and higher average students, but not extremely low and 
high students, it is surprising how clearly the teachers differentiated between 
the two levels of achievement. The influence of achievement data on preser- 
vice teachers’ program recommendations extends earlier findings of the 
influence of achievement on teacher expectations (e.g., Benz et al., 1984) and 
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teacher decisions about placement for elementary students (e. g., Hutchinson 
& Hemingway, 1984). 

In summary, the findings of the present study suggest that teachers’ 
program advice to students may be influenced by gender, ethnicity, and 
achievement. If teachers frequently advise adolescents about program selec- 
tion in the absence of counselors (Lee & Ekstrom, 1987), then students who 
are influenced by this advice may make course selections that inappropriate- 
ly limit their career aspirations and accomplishments. Teachers need to be 
informed of the counseling influence they may exert on their students and 
the potential for bias based on student attributes such as gender and eth- 
nicity. Advice given to students may be a factor contributing to occupational 
stratification. Research is needed to continue the investigation of variables 
that mediate between attributes such as ethnicity (Clifton & Roberts, 1988) 
and gender (Farmer, 1987) and outcomes like occupational aspirations and 
accomplishments. 


Note 


Correspondence regarding this article should be sent to Nancy Hutchinson, Faculty of Educa- 
tion, Queen’s University, Kingston, Ontario, Canada, K7L 3N6. 
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THE EDITOR’S PAGE 


Attacking the Two Solitudes 


During the past 10 years, research in education has undergone dramatic 
change. The popularity of operationalism has been yielding to a more respon- 
Sive, interpretive way of studying educational phenomena. New journals are 
springing up to accommodate researchers who pursue their work in these 
ways with the result that there is a developing sense of methodological 
isolation. Unwittingly, The Alberta Journal of Educational Research may be 
contributing to the “two solitudes” of educational research. Based on the 
kinds of articles that are submitted, I believe that AJER is seen as an outlet 
for the numerical approaches to educational research. For several reasons 
this is unfortunate. In this longer-than-normal editorial, I would like to 
propose a new agenda for AJER. 

Education as a social institution involves millions of Canadians and bil- 
lions of dollars, yet there are fewer than 500 people who are engaged in 
continuous, serious, and systematic study of the field. Moreover, their efforts 
are divided into dozens of subspecialties, so that at any time the research 
attention being paid to a particular issue is very small. To further divide this 
on the grounds of method has the effect of partitioning an already too small 
work force. 

The nature of competence is such that whether you are an ethnographer, 
a feminist interpreter, or a factor analyst, the apprenticeship that must be 
served is so long and intensive that it seems to preclude multimethod com- 
petence within a single researcher. In addition, the best researchers in any 
field are usually passionate believers in the correctness of their own stand, 
and as human beings they are naturally drawn to like-minded souls. Conse- 
quently, it is not surprising to find lines of research proceeding in parallel 
tracks that are insulated from the influence of each other. Research on 
classroom instruction is a splendid example. Despite this, the consumer of 
research has much to gain from an awareness of the literature that derives 
from various frameworks. 

Being confronted by different perspectives encourages debate and rein- 
tegration within a field. In my own area, educational measurement, the 
criticisms of social philosophers and curriculum theorists have led to new 
topics of research interest (for example, test bias) and to new ways of 
thinking about old ones (for example, validity). In turn the measurement 
people are contributing to educational policy and teaching practice. The 
debates are not always peaceful, and they may not always convince in- 
dividual proponents of a particular perspective. The history of educational 
inquiry tells us that no group has exclusive rights to wisdom no matter how 
convincing their arguments may appear in the immediate context. Evidence, 
analysis, criticism, and experience inevitably produce cracks in the position. 
To be effective, the debate among competing ideas must be heard by those 
who are involved in doing the research and by those who wish to use it. It is 
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simply not functional to isolate studies of educational phenomena on grounds 
of method. 

AJER can facilitate the debate in two ways. First of all we (the board and 
I) invite researchers from all perspectives to send us the products of their 
investigations. In the past I may have been less than careful in finding 
reviewers who had the necessary breadth of experience and the tolerance of 
perspective to provide a helpful review for a particular paper. The outline 
provided for use by reviewers has often been “not quite appropriate” for the 
task, with the result that a message may have been sent to the effect that 
AJER is interested only in a particular kind of article. 

Dimensions for assessment will now stress context, conceptualization, 
clarity, coherence, and contribution. ‘The questions will be: What is the 
purpose of this paper? Is there a point of view being developed? Are the 
arguments persuasive? Do the conclusions or positions follow from the argu- 
ments, analyses, and evidence? Are they expressed in a way that the reader 
can understand and respond to even though he or she may not be in the area 
of specialization? Does the paper make a contribution? Does it provide some 
insight into the nature of educational phenomena? 

Previously there was great emphasis on specification of research design, 
method of data collection, and type of analysis. Now this falls within the 
general notion of validity of argument. If a writer makes a claim based on 
experimental evidence, then the logic of the study must support the claim. If 
the claim is being made on the basis of conceptual analysis, then the logic of 
that analysis will be examined. Reviewers will be invited to ask the questions: 
What does the writer mean, and how (or why) does she or he come to those 
conclusions? 

The second way in which AJER can encourage a richer approach to 
understanding educational research is to encourage more special issues. At 
the moment, each issue contains a collection of articles from many domains. 
This is a necessary consequence of a relatively small number of researchers 
working in many fields. However, by providing an opportunity for a single 
outlet for several connected papers, we can provide better service to both the 
readers and the writers. Articles in single topic issues will undergo an inde- 
pendent review process, and then a few members from the editorial board 
will be asked to examine the set of articles for coherence. It is hoped that 
organizers of single topic issues will use this opportunity to bring together 
researchers from several perspectives. 

From the reader’s point of view, these changes should be beneficial. 
Articles from different methodological persuasions will provide a more ac- 
curate portrayal of the diversity of our field. Single topic issues will offer an 
efficient way of finding out about a particular line of educational thought. 
Because the research community is so thinly spread, every effort will be made 
to aim each article at the knowledgeable (but not necessarily expert) educa- 
tional scholar. The members of the editorial board hope that these changes 
will encourage debate and facilitate the drift of ideas across boundaries of 
method. The reward will be sharper arguments and deeper insight. 
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A Process-Based Approach to the 
Remediation of Spelling in Students 
with Reading Disabilities 


Aspects of Das and Conway’s (1986) remedial program CAP (Coding, Attention, and 
Planning training) were combined systematically with a spelling program designed to 
reflect the same global cognitive processing strategies. Daily intervention, consisting of 
training in both the global tasks and the specific spelling tasks, was provided for 10 
learning disabled students with particular deficits in spelling, to a total of 20 hours of 
remediation. Ten matched, learning disabled students received 20 hours of the normal 
classroom remedial spelling program. Following remediation the experimental group per- 
formed significantly better than did the control group on the K-ABC sequential subtests, 
selected CAS planning subtests, and the Test of Written Spelling (Larson & Hammill, 
1986). Results are discussed as supportive of an appropriate and viable remedial option for 
spelling with learning disabled students. 


Some children are just born poor spellers. (Flaro, 1985, p. 4) 
The ability to spell well is a gift from God. (Lerner, 1981, p. 337) 


The above suggestions typify the attitudes of many educators regarding the 
ability to spell. The implication is that spelling ability is innate, and as such 
the onus on educators to influence the success of their students in this area is 
certainly lessened. It has been noted by several researchers that teachers 
spend relatively little time teaching spelling, particularly when compared 
with an area such as reading, and that often techniques for teaching spelling 
are disregarded even when they have been empirically validated (Gerber & 
Hall, 1987; Wallace & Larsen, 1978). Yet spelling ability, or lack of such 
ability, can influence a child’s total school experience, affecting as it does 
one’s ability to communicate in written form (Hammill & Noone, 1975; 
Wallace & Larsen, 1978). 

The close relationship between reading and spelling is reflected by the fact 
that most children who experience reading difficulties invariably have dif- 
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ficulty learning to spell (Gerber, 1984; Rieth, Polsgrove, & Eckert, 1984; 
Wong, 1986). Such children with specific learning disabilities are particularly 
vulnerable to the emotional burden of their academic deficiencies, and the 
related lowering of self-esteem, in that “incorrect spelling often creates an 
unfavorable impression beyond its true significance” (Otto & McMenemy, 
1966, p. 203). It thus seems vital that attention be given to remediation of a 
skill which may otherwise persist as a serious problem with children who are 
already struggling with learning problems in areas such as reading. 

Initially, traditional spelling programs were the only viable option avail- 
able for the reading disabled student. The majority of these programs 
reflected a linguistic theory wherein emphasis was placed on the teaching of 
phonology, morphology, and syntactic rules (Hammill, Larsen, & McNutt, 
1977). For most reading disabled students, these programs proved to be 
ineffective. More recently there have been several attempts to gain a 
“process-based” understanding of spelling and its remediation in learning 
disabled students (Englert, Hiebert, & Stewart, 1985; Gettinger, 1985; Kauf- 
man, Hallahan, Haas, & Boren, 1978; Wong, 1986), but the results to date 
remain equivocal. 

The objective of the present study was to design and test instructional 
procedures that would facilitate spelling ability in students with reading 
disabilities. The procedures were based on the cognitive processing model of 
Das and colleagues at the University of Alberta (Das, Kirby, & Jarman, 1975, 
1979). 

This “information-integration model” involves an integration of neurop- 
sychology and information processing psychology, and within the model the 
conceptualizations of Soviet psychologist Luria (1970, 1980) have been used 
to describe three global but interactive cognitive processes: coding, attention, 
and planning. Coding involves the manner in which information is organized 
by the brain, successively and/or simultaneously, and is largely accomplished 
within the temporo-frontal and occipito-parietal cortex. Empirical evidence 
for successive and simultaneous processing has been generated in a large set 
of studies in which the performance of normal and disabled children on a 
diverse set of cognitive measures has been factor analyzed (Das et al., 1979). 
Similar processes have also been described by cognitive psychologists, for 
example, Neisser (1976). The attention component of the model includes 
arousal, a subcortical function, as well as modulation of arousal regulated via 
the frontal cortex in concert with subcortical processes. Planning, the third 
component, is broadly defined as the generation, selection, and execution of 
programs, either conceptual or behavioral. Kirby (1984) has identified at 
least four factors within the planning domain including search and metacog- 
nition. 

Based on the information integration model, Das and his associates have 
developed a cognitive approach to remediation founded on three premises: 
(a) children learn most effectively by dynamically interacting with adults and 
peers; (b) remediation is most effective when done from a theoretical 
perspective in which the cognitive processing of children is related to the 
processing requirements involved in learning academic skills; and (c) trans- 
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fer of learning will be maximized if general, nonthreatening tasks are com- 
bined with academic bridging tasks. Support for the first premise of dynamic 
interaction is provided in the writing of Vygotsky (1962) and Feuerstein 
(1979) and in the research of Tizard and her colleagues (Tizard & Hughes, 
1984). Empirical support for the second premise of the remediation model, 
that global cognitive processes (successive and simultaneous coding, atten- 
tion, and planning) related to academic skill processes can be used to effec- 
tively remediate skill deficiencies, is provided by a series of studies done over 
the years by Das and associates (Kaufman & Kaufman, 1979; most recently 
through development and use of the Coding, Attention and Planning [CAP] 
remediation program by Das & Conway in 1986) and by independent inves- 
tigators (Garofalo, 1986; McRae, 1986). Finally, support for the third 
premise of the remedial model, that academic bridging tasks may enhance 
the transfer of global processing to academic skills, is found in the writings of 
Douglas (1980), Kirby (1984), and Leong (in press). 

Based on the above premises, the model for spelling remediation within 
the present study was developed through an extension of aspects of the global 
CAP program (Das & Conway, 1986). In particular, specific bridging tasks 
involving spelling were created to accentuate transfer of successive coding 
skills to spelling ability. Because, as described previously, both successive and 
simultaneous coding, and attention and planning dimensions are implicated 
in all cognitive tasks, it was of interest to examine marker task results in 
each area as pretest/posttest measures. Unfortunately, at the time of the 
present study the “attentional” marker tasks were under extensive revision, 
and the focus was limited to an examination of scores for coding and plan- 
ning. Actual remedial training focused on successive coding, following indica- 
tions that initial spelling relies heavily on successive strategies (Brailsford, 
Snart, & Das, 1984), but it was felt that related improvements in other areas 
of cognition were possible, and worth noting. The remedial training aspects 
of attention and planning were addressed through the specific and directed 
verbalizations with students. Strategy training for the reading/spelling dis- 
abled students involved five CAP tasks which focused on successive process- 
ing, and five spelling tasks which were developed to reflect the principles and 
skills underlying each CAP task. Following remediation it was hypothesized 
that the experimental students would exhibit substantial improvements in 
spelling ability and improvements on successive coding tasks. Pretest and 
posttest results for planning and simultaneous coding were examined on an 
exploratory basis. 


Method 

Subjects 

Twenty learning disabled students in two special classes in the Edmonton 
Public School system served as participants in this study. The criteria for 
inclusion in a learning disabilities class within the Edmonton Public system 
consist of an IQ within the average range or above (WISC-R), and academic 
achievement in at least one area of more than two years below the expected 
level on standardized measures. 
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The students’ ages ranged from 8 to 12 years, and WISC-R IQ means for 
each of the two classes (V=10 per class) were within the average range. 
English was their native language, and they did not have either visual or 
auditory impairments. Means for age, IQ, and Schonell Spelling Inventory 
for each class are presented in Table 1 (¢-tests comparing these means in each 
case revealed no differences between the two groups). Schonell test results 
were readily available through school files and provided initial data for group 
selection, though the more detailed analyses of spelling skills provided by the 
Test of Written Spelling (Larson & Hammill, 1986) were utilized in 
pretest/posttest comparisons. The students were also two or more years 
behind their age peers in reading achievement. One class of 10 students 
served as a control group who received the typical classroom remediation in 
spelling, while the other class received an equivalent number of hours of the 
experimental remedial program and no other intervention in spelling during 
this time period. When students from either group were absent they were 
given “make-up” hours of the appropriate spelling remediation. In both cases 
the “instructional word list,” comprising those words used for remedial 
purposes, consisted of 39 words from grades 1 through 5 which had been 
carefully selected by the teachers from the lists constructed by Thomas 
(1974), and which were felt to be representative of words most frequently 
misspelled by Alberta schoolchildren. 


Pretest and Posttest Battery 


The pretest battery was administered during January and the posttest bat- 
tery during April of the same year. In each case the testing was conducted in 
a quiet setting in the child’s school. Tasks chosen as pretest and posttest 
indicators included spelling tasks which were felt to be valid and reliable 
indicators of spelling skills in elementary-aged students in Alberta, and 
marker tasks for coding and planning. The latter marker tasks in the plan- 
ning area are those which Das & Conway (1986) also used in pretest and 
posttest studies of the CAP remediation program of global cognitive func- 
tions. Though the planning marker tasks and marker tasks for coding and 
attention are currently under development by Das and colleagues as the 


Table 1 
Means and Standard Deviations for Age, |Q, and Schonell Spelling Inventory 
for Experimental and Control Groups 


Control Experimental 

Group Group 
Age in months 127.70 (7.24) 128.10 (15.73) 
IQ 92.50 (9.06) 93.20 (8.32) 
Schonell Spelling 2.50 (0.67) 2.30 (0.81) 


Inventory 


(Grade score) 
Se a ee ote veto 0k Bea a ee 
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Cognitive Assessment System (CAS, Naglieri & Das, 1987, 1988, in press), a 
decision was made to utilize tests for successive and simultaneous processing 
which had previously been published within the same theoretical framework 
and were better known at this time (Kaufman & Kaufman, 1983). Pretest 
and posttest tasks included the following: 


Spelling Tests 
1. & 2. The Test of Written Spelling (TWS-2, Larson & Hammill, 1986) 


was administered to the students in a group setting. This test is 
divided into two subtests to accommodate the theory that a child 
learns to spell in two ways. First, the child learns to apply rules and 
generalizations, and second, the child must rely on memorization 
skills. The former is instrumental when learning words that con- 
form to the rules (i.e., “predictable words”) while the latter is neces- 
sary to spell words that do not (i.e., “unpredictable words”). The 
students in this study were given both of the TWS-2 subtests. 

The TWS-2 was standardized on an American sample of 3,805 child- 
ren across 15 states. The 50 words that comprise each of the sub- 
tests were carefully selected to ensure that they were actually 
taught in schools. The words up to grade 8 appear in five commonly 
used American basal spelling series and the words above grade 8 ap- 
pear on word frequency lists. Within the present study, raw scores 
for each student were converted to standard scores for predictable 
words, and unpredictable words. It is also noteworthy that the in- 
structional word list used for both remedial groups did not contain 
any of the spelling words from the TWS-2. 

The administration of the TWS-2 involved the examiner first saying 
the word in isolation, then using the word in a sentence, and finally 
saying the word again in isolation. The subjects were required to 
write the dictated words. 


Cognitive Assessment System (CAS) Tests (Naglieri & Das, in press) 


3. 


Visual Search is one of the Planning subtests of the CAS. It com- 
prises 20 search tasks grouped under either the Automatic Search 
or Controlled Search component. The child’s task is to find a pic- 
ture, number, or letter identical to one located in a box in the center 
(called the Target) somewhere within the group of stimulus items 
(called the Field). The score is the time in seconds for the child to lo- 
cate the Target embedded in a field of similar items (picture among 
pictures, number in numbers, letter in letters) or dissimilar items. 
Automatic Search time is the sum of times for searching where the 
Target differs in kind from the Field items (e.g., a picture among 
letters) whereas Controlled Search time refers to those where Tar- 
get and Field items are the same. Automatic Search is essentially a 
measure of attention as the target “pops up” from the field, 
whereas in controlled or planned search the child has to make a 
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deliberate effort to locate the target. Alpha reliability of the test is 
.64 based on 135 children in grades 3 through 5. 

Crack the Code is a Planning subtest from the CAS. In this subtest 
the student’s task is to determine the correct sequence of a number 
of colored chips when given a limited amount of information. The 
subject is given two or three trials with feedback to determine the 
one correct order of chips for each of the 7 items. The subtest is or- 
ganized into 7 items with 2 or 3 trials per item and a 3-minute limit 
per item. The score is determined considering number of trials 
necessary to produce the correct sequence. Alpha reliability is .68 
based on 140 children in grades 5 and 6. 

Planned Connections is a CAS Planning subtest. The child’s task in 
this subtest is to connect, using a regular pencil, a series of num- 
bered boxes in correct numerical sequence as quickly as possible. 
There are two kinds of items in this subtest: those that include num- 
bers only, and those that include numbers and letters. The latter 
items involve connecting the boxes using numbers and letters alter- 
nately (1-A-2-B-3-C etc.). The overall score for all items is the time 
in seconds required to complete them. Alpha reliability is .64 based 
on 70 children in grades 5 and 6. 


Subtests from the Kaufman Assessment Battery for Children (K-ABC) 
(Kaufman & Kaufman, 1983) 


6. 


10. 


1h 
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The Hand Movements subtest of the K-ABC is a Sequential process- 
ing task in which the child is required to reproduce a series of hand 
movements in the same sequence as the examiner. As with all the 
K-ABC processing tasks, the raw score is converted to a scaled 
score. The reliability of this test and the others which follow are 
given in the K-ABC manual. 

Number Recall is a K-ABC Sequential task, in which the child at- 
tempts to repeat a series of numbers in the same sequence as the ex- 
aminer. 

Word Order is a K-ABC Sequential task which involves the child 
touching a series of silhouettes of common objects in the same se- 
quence as the examiner has verbalized the names of the objects. 
(More difficult items include an interference task between the 
stimulus and response.) 

Gestalt Closure is a Simultaneous processing task from the K-ABC. 
The child is required to name an object or scene presented as a par- 
tially complete “inkblot” drawing. 

Triangles is a K-ABC Simultaneous task in which the child as- 
sembles several plastic triangles into patterns to try to match a 
series of models. 

Matrix Analogies is a K-ABC Simultaneous task, requiring that 
children select a meaningful picture or abstract design which best 
completes a visual analogy. 
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12. Spatial Memory is a K-ABC Simultaneous task which involves 
recall of the placement of a set of pictures on a page which was ex- 
posed briefly. 

13. Photo Series requires students to place photographs of an event in 
chronological order, and is one of the K-ABC Simultaneous tasks. 
The K-ABC was standardized on more than 4,100 administrations. 
Split-half reliability was established with the use of the Rasch- 
Wright model. The mean value for coefficients of internal consisten- 
cy for 12 of the 16 subtests was .80 or above. Support for the 
construct validity of the K-ABC mental processing subtests was es- 
tablished through a correlational study between K-ABC and Factor 
scores on the Das-Kirby-Jarman Successive-Simultaneous battery 
(see K-ABC Interpretive Manual). 


Intervention Program 

Five global CAP tasks which had loaded most clearly on successive processing 
in factor analytic studies were selected and five spelling tasks were created, 
modeled after each of the CAP tasks. Representative examples of two of the 
global tasks are presented in the Appendix, along with the related spelling 
task in each case. One important component of the remedial training was 
having students talk about their approaches to the tasks, how they might 
change an approach to have it “work better,” and so on. Within such discus- 
sions with the teacher the growth of planning ability was predicted. 

The remedial sessions for the intervention group took place on a daily 
basis for each child, according to a prepared schedule, and located at stations 
in the student’s classroom. A total of 20 hours of global and content-specific 
training was given to each child in the experimental class, while children in 
the control class received an equivalent number of hours of the “I-Can-Spell” 
program (Rodgers, Covell, & Slade, 1978). In this approach students were 
presented with words from the instructional word list at their respective 
ability levels, in a pretest-study-posttest fashion. Fach word list was accom- 
panied by workbook activities prior to posttest. 

The daily remediation session in the intervention program involved train- 
ing at two stations. Each station trained a different CAP task. A student 
would receive 5 minutes of training on a CAP task and then 10 minutes on 
the associated spelling task at each station (as in the control group the 
spelling words were those from the instructional word list). Following the 
training at both stations a cumulative spelling test was administered bring- 
ing the total time of the session to 35 minutes. The stations were operated by 
the classroom teacher and the classroom aide. 

The remedial training was scheduled for the mornings. The students were 
on individual programs and worked independently on assignments until they 
were called for their remedial session. After the remedial session, students 
returned to their individual assignments. Each day the students were 
provided with schedules for their morning assignments. The instructional 
component of the students’ school day was scheduled for the afternoons. 

The students’ daily performance on the CAP and spelling training was 
recorded. Specific record forms were used for each activity. The information 
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obtained from the CAP record forms provided the basis for establishing a 
starting point for training the student the next time the task was used. The 
spelling record forms provided the teacher with diagnostic information in 
terms of how particular students were misspelling words, as well as an 
ongoing indication of words which were spelled correctly. 


Results and Discussion 

Of vital importance in this study was an examination of the actual improve- 
ment in spelling achievement. This emphasis was reflected in the fact that 
two thirds of remedial training time was spent on spelling tasks, with one 
third spent on training with the global.tasks. An expectation for concurrent 
improvement on the global tasks existed whereby specific training would 
enrich global processing in addition to task-related processing. As a limita- 
tion of the present study, it must be noted that results may be directly 
generalized only to populations within segregated classes for students with 
learning disabilities, and that further research will be necessary to determine 
broader implications of the method. 

Student scores on the Test of Written Spelling were analyzed using a 2 
(Group) x 2 (Pretest/Posttest) x 2 (Predictable/Unpredictable words) 
ANOVA, and group means are presented in Table 2. Main effects for 
Pretest/Posttest, (F1,183=68.2; p<.001), and Word Type, (F1,18=70.7; 
p<.001) reflected the superior performance overall at posttest, and for pre- 
dictable words. A significant Group x Pretest/Posttest interaction occurred 
(F'1,18=14.2; p<.001) with spelling scores at posttest significantly higher for 
the experimental students thus providing support for the effect of the re- 
medial program. It is also of interest to note that, although the small N per 
cell contributed to the lack of a significant Group x Pretest/Posttest or 
Group x Pretest/Posttest x Word interaction in the ANOVA, the means in 
Table 2 certainly suggest that the experimental students were better able 
than the control students to handle unpredictable words at posttest. Because 


Table 2 
Mean Scores on the Test of Written Spelling (TWS-2) 
for Experimental and Control Students 


Pre Post 

Predictable 9.60 13.60 
Experimental Words (SD = 5.38) (SD =6.10) 
(N= 10) 

Unpredictable 6.80 10.30 

Words (SD =3.77) (SD = 4.27) 

Predictable 10.40 12.50 
Control Words (SD,=2.72) (SD =3.34) 
(N= 10) 

Unpredictable 7.00 140 

Words (SD =2.31) (SD = 3.02) 


a ee Ee eee 
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predictable words conform to rules and generalizations, while unpredictable 
words demand that the student focus on and analyze the letters sequentially, 
it might be further suggested that the latter skill may be enhanced through 
successive strategy training. 

Results of the present remedial program hold promise for assisting dis- 
abled learners in their ability to spell. Anecdotal evidence from the 10 stu- 
dents in the experimental group, when compared with their 10 peers in the 
control group, suggests some positive and more broad-based effects of the 
remedial program. In all cases, the experimental students gained confidence 
in their ability to write as their spelling skills increased, and their written 
expression as analyzed through actual work sample comparisons was mark- 
edly improved. The fact that the negative “set” toward writing was lessened 
allowed these students to practice their writing skills, and of course one 
success built on another! The experimental students also became more aware 
of their own learning and became “active learners” during the remedial 
sessions, which is evident not only in the planning posttest results described 
later, but also in their more optimistic and flexible approaches to problem 
solving in the classroom in general. 

To examine the results for global processing and provide direction for 
future studies having larger sample sizes, a series of t-tests was performed on 
Pretest-Posttest means for each group (means are presented in Table 3). 
Significant improvements were revealed for the experimental group in se- 
quential coding (t1,9=3.64, p<.005), and for both the experimental 
(t1,9=3.44, p<.010) and control group (t1,9=3.84, p<.005) in simultaneous 
coding. Certainly a case can be made for an increase in cognitive coding 
ability over time for all students, based on maturity and living experience, 
academic experience, and so on. However, there seems to be a more complex 
issue at hand when one examines sequential or successive processing. Past 
research has suggested that successive coding is particularly implicated in 
early reading skills such as basic decoding (Brailsford, Snart, & Das, 1984; 
Cummins & Das, 1977), and that even intellectually advanced students with 
learning difficulties may require specific successive training prior to achiev- 
ing these basic skills (Snart, Das, & Mensink, 1988). The active intervention 
with the experimental group, using successive coding tasks, was apparently a 
factor in the differential improvement of sequential ability for these stu- 


Table 3 
Mean Cognitive Coding Scores for Experimental and Control Students 


Sequential Score Simultaneous Score 

Pre Post Pre Post 
Experimental 21.50 27.20 46.50 52.00 
(N = 10) (SD = 3.66) (SD = 5.20) (SD= 10.65) (SD=12.28) 
Control 26.20 28.40 46.10 50.30 
(N = 10) (SD=5.81) (SD =4.17) (SD=10.52) (SD= 10.60) 
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Table 4 
Mean Scores for Planning Marker Tasks at Pre/Post Test 


Task Experimental Group Control Group 
Pre Post Pre Post 

Visual Search 34.47 28.73 35.08 29.10 
(Automatic) (SD =8.15) (SD =8.09) (SD=11.16) (SD=12.70) 
Visual Search 121.78 102.94 118.75 121.15 
(Controlled) (49.34) (19.48) (25.95) (29.88) 
Planned 136.00 100.67 130.06 114.08 
Connections (52.71) (43.54) (36.62) (35.13) 
Crack the 1.80 3.80 3.10 3.80 


Code (2.20) (1.75) (1.79) (1.81) 


dents, and the impact of this improvement is emphasized when we examine 
actual spelling achievement. 

Planning ability within the information integration model has generally 
been understood as a superordinate ability relating to a student’s foresight 
and flexibility in problem solving. Taking into consideration again the small 
N per cell, ¢-tests of Pretest-Posttest means on planning marker tasks were 
examined for each group. Significantly better performance at posttest was 
found for the experimental students on the Visual Search (Automatic) task 
(t1,9=3.42, p<.01) and Planned Connections task (¢1,9=2.73, p<.05), with 
none of the planning tasks showing pretest/posttest differences for the con- 
trol students (means are presented in Table 4). Automatic search requires 
little planning; it is a test of attention. Of course, one can only speculate 
based on these results, but it seems that remedial training, encompassing as 
it did the verbalization of students in problem solving, may have contributed 
to an improved attention and planning ability overall. An initial indication 
such as this opens the door to continued research in this area. 

In summary, the notion of spelling ability (or disability) as an immutable 
factor in students is certainly challenged by the present data which support a 
remedial option for disabled students. It appears initially that global plus 
bridging tasks focusing on successive coding and spelling could be successful- 
ly incorporated into whole class instruction for students with learning prob- 
lems, and that they could impact positively on spelling achievement. Perhaps 
even more importantly, this study provides direction and optimism for future 
research into the use of such a model, utilizing nonacademic global tasks plus 
bridging tasks to an academic area, in subject areas beyond spelling (e.g., a 
simultaneous coding emphasis plus bridging tasks in reading comprehen- 
sion). Further, such a remedial model may have direct application within a 
resource room situation, or even within a regular classroom with a preven- 
tive focus. The fact that strategies rather than merely academic content are 
actively taught, and that attention and planning abilities receive considera- 
tion within the context of the “delivery” of the remedial teaching, makes this 
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intervention more motivating for students and more likely to generalize to 
other academic pursuits of the student. 
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Appendix 


Descriptions of Two CAP Global Tasks 
and Related Spelling Remedial Tasks 


CAP Task 10 Bead Threading 


Strategy: Successive. 

Objective: To thread a bead pattern by recall following either teacher 
demonstration or card study. 

Instructions: 

1. A “bead pole” is used: This has a cardboard cover around it and a win- 

dow cut out in the cardboard cover. The teacher drops beads in a 

predetermined order, ensuring that the beads can be seen briefly as they 

pass through the window. 

Learner copies the pattern using his or her own frame. 

Correctness of sequence is checked by lifting the cover off the frame. 

Student should repeat the task if incorrect (provide more cues). 


Teacher records response (1 for correct, 0 for incorrect). 
or 


Oe ee eS 


1. Teacher holds up a card with a picture of beads on a bead pole for the 
student to study. 


Card is removed. 

The student then attempts to duplicate the pattern observed. 

Student then checks the card for accuracy. Teacher records the response 
(1 for correct, 0 for incorrect). 

5. Student should repeat the task if incorrect. (Discussion of clues and 
strategies should precede next attempt.) 


go cert 
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Spelling Task to Follow CAP Task 10 
Instructions: 


1. Teacher presents a card with letters arranged on it, says, “This word is 
,” and turns the card over immediately saying, “Use the letters in 
front of you to make this word” (student places letter cards in a card 
holder). Teacher records the response and asks the student to mix up 
the letters. 

2. Teacher presents a window chart to the student. Each window has a 
strip of paper with a letter on it. The teacher says, “Watch carefully.” 
The teacher then pulls the strips through the windows one at a time, 
saying the letter name. The letter is exposed for no longer than 5 
seconds. Student is then asked to arrange the letters in front of him in 
the card holder as he remembers seeing them. Teacher records the 
response. Student checks his response by exposing the letters in the win- 
dows. Each letter is left exposed to allow student to check. Student cor- 
rects errors as they are discovered. 

3. Teacher takes the chart from the child and holds it up for the child to 
see that the letters are still exposed. Teacher then says the name of each 
letter and pulls the letter out of sight. Teacher says the whole word after 
the last letter has been named and pulled out of sight. The student is 
then asked to place the letter cards in the holder in the correct order. 
Teacher records the response. The student is then given the window 
chart to check his or her response. Student checks his or her response by 
exposing the letters in the windows. Each letter is left exposed to allow 
student to check. Student corrects errors as they are discovered. 

4. Repeat step 3. 

5. Repeat step 1. 


CAP Task 16 Telephone 

Strategy: Successive. 

Objective: To dial from memory telephone numbers of increasing length. 
Instructions: 


1. The teacher reads a digit number sequence (allow 1 second delay be- 
tween each digit). 

2. Learner then dials this sequence on a telephone. Teacher records 
response (1 for correct, 0 for incorrect). 

3. Repeat sequence if incorrect—discuss ways of remembering. 

4, Discuss student’s correct response. 


Spelling Task to Follow CAP Task 16 
Steps to Follow: 


1. Student finds the correct envelope for the spelling word (envelopes are 
numbered). Inside the envelope are the materials required: 
— language master card (side 1 and side 2); 
— letter cards inside a small white envelope (15 letters in each en- 
velope); 
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~ 8 yellow sheets numbered 1, 2, 3 respectively. Student’s spelling word 
is recorded on the language master card. On side (1) the word is said 
and then used in a sentence. On side (2) the word is spelled, that is, 
Side (1) turkeys—turkeys are birds 
(2) t-u-r-k-e-y-s. 
Student takes everything out of the envelope. 


. Student is instructed to listen to side one of the language master card. 


The student is to print the spelling word he or she heard on yellow sheet 
numbered (1) and place it in the box beside him or her. Student is 
reminded to check that his or her name is on the sheet. (The sheets have 
been predated.) 


. Student is to empty the letters from the white envelope and arrange 


them in a line. 


. Then the student listens to side 2 of the language master card. 
. The student has extra letters. He or she must choose the correct ones to 


make the word after he or she has heard the spelling. The letters are 
spread out on the table. He or she must choose from the selection—10% 
of the spelling words will have misinformation, that is, not all of the re- 
quired letters will have been included—the student will be encouraged 
to solve the problem. 


. Student then listens to side 2 again to check his or her arrangement. If 


there are errors student fixes them and then checks by listening once 
more. 


. Student mixes up the letters and repeats steps 4 through 6 two more 


times. 


. Now the student writes his or her arrangement for the spelling word on 


the yellow sheet numbered (2). The student then places this sheet in the 
box beside him or her. 


. The student puts everything back in its place and then writes the spell- 


ing word on the yellow sheet numbered (3) and then places it in the box. 
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A Report on Learning Disabled Children’s 
Use of Lab-Based Word Processing Versus 
Pencil-and-Paper for Creative Writing 


To determine if learning disabled (LD) children would be able to utilize lab-based word 
processing (W-P) to enhance writing, a six-month study of writing performance was con- 
ducted with four LD children and matched normal achievers (NA) (age range 11 to 12 
years). The W-P benefited both groups in terms of increased drafts and overall output. In 
formatting (paragraphing, spacing, etc.) the LD did not perform as well as NA children 
during W-P sessions. For both groups word processing did not lead to (1) quantitative 
differences between first and final draft length; (2) improvement on the Test of Written 
Language; (3) increased revisions involving meaning changes, spelling, punctuation, tense, 
number, abbreviations, and so forth. Thus the results indicate the learning disabled use 
lab-based word processing in a manner generally comparable to the normal achievers, and 
the positive effects are limited to quantitative rather than qualitative changes. 


In this study we explored the viability of using a word processing lab for 
writing with learning disabled elementary-school children. We had pre- 
viously documented that normal achieving elementary and junior high school 
level children were able to use a word processing lab quite effectively. We 
found that word processing led to improvements in writing productivity and 
revision when compared with paper-and-pencil performance (Morton, 1988; 
Morton, Lindsay, & Roche, 1989). 

Because writing can be a major problem for the learning disabled (Lerner, 
1976), and remedial progress is often slow and taxing (Johnson & Myklebust, 
1967; Reid & Hresko, 1981), the word processor has been advanced as 
potentially important for this group (MacArthur & Shneiderman, 1986; 
Sapona, Lloyd, & Wissick, 1986). In view of the logical appeal of word 
processing and the favorable results emerging with normal achievers (Mor- 
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ton, 1988; Morton et al., 1989), we extended our research to include learning 
disabled children. Consistent with our previous research we predicted that 
word processing would enhance their writing in terms of (1) drafts produced, 
(2) number of words written on final drafts, (3) total word output, and (4) 
revisions made when compared with paper-and-pencil sessions. 


Method 

Subjects 

Four learning disabled children (two males and two females; age 
mean=11.85 years, SD=0.65) were matched on sex and age with four nor- 
mal achieving students (age mean=11.90 years, SD=0.42). The learning 
disabled were intellectually average (WISC-R, Full Scale IQ=88.3, SD=4.1) 
and at least two years behind academically. They had been identified by a 
Special Education, Identification, Placement, and Review Committee and 
placed in special self-contained classrooms. Test scores on the Test of Writ- 
ten Language (TOWL, Hammill & Larsen, 1983) placed the learning disabled 
within the low-average range (standard score mean=8.2; average range is 7 
to 13); the normal achievers were approximately one standard deviation 
higher (mean=11.1). 


Procedure 


We used a virtually identical procedure to that reported in Morton et al. 
(1989). The students used the microcomputer for creative writing (based on 
a Whole Language approach) for three months and used paper-and-pencil for 
a comparable three-month period with appropriate counterbalancing for 
order effects. Six microcomputers and two printers were set up in an empty 
classroom and students used this lab setting independently for 30-minute 
sessions of writing. Writing was monitored from the regular classroom by 
teachers and lab assistance was provided by a cooperative education student 
on work assignment. Three TOWL subtests (Word Usage, Vocabulary, and 
Thematic Maturity) were used for pretest and posttest comparisons. 

To determine productivity (i.e., quantitative changes) in both pencil-and- 
paper and word processing settings, we recorded (1) number of drafts 
produced, (2) number of words in each draft, and (8) total output of each 
group in each setting. To evaluate qualitative changes we used a taxonomy 
modeled after Faigley and Witte (1984) on the first 30 words in each revised 
draft (Morton et al., 1989). We considered lexical revisions (i.e., additions, 
deletions, etc.) at three levels: First Order (meaning preserving), Second 
Order (meaning-altering but gist-preserving), and Third Order (meaning-al- 
tering and gist-altering). We considered Infralexical revisions also at three 
levels: First Order (changes in spelling and punctuation); Second Order 
(grammatical changes in tense, number, etc.); and Third Order (structural 
changes in format and paragraphing). 


Results 
Writing Productivity 


. Number of drafts. A two-way (2 x 2) ANOVA with Group (learning 
disabled, normal achieving) and Setting (word processing, pencil-and-paper) 
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as the independent variables was computed for number of drafts produced. 
Setting was a within-subjects variable. There was a main effect for Setting, 
F(1, 6)=17.32, p<.006, with subjects producing more drafts during the word 
processing sessions (mean=2.79) than the paper-and-pencil sessions 
(mean=1.83). Also, there was a main effect for Group, F(1, 6)=13.16, p<.01, 
with the learning disabled producing more drafts (mean=2.71) than the 
normal achieving (mean=1.9). There were no interaction effects. Means and 
standard deviations for productivity are presented in Table 1. 

Total words produced. For total output a two-way (2 x 2) ANOVA (Group 
by Setting) was computed for number of words produced. There was a main 
effect for Setting only, F(1, 6)=13.40, p<.01, with subjects producing more 
during word processing (mean=2394.87) than paper-and-pencil sessions 
(mean=1413.36). A marginal two-way interaction for Group and Setting, 
F(1, 6)=5.57, p=.056, indicates a tendency for normal achievers to produce 
more overall on the word processor. Post hoc simple effects tests show that 
the normal achievers are producing more than the learning disabled in the 
word processing sessions only, ¢(6)=3.36, p<.02. 

Draft lengths. A three-way (2 x 2x 2) ANOVA (Group by Setting by Draft) 
was computed for length of first and final drafts. There were no main effects 
or interaction effects. 

A marginal three-way interaction for Group, Setting, and Draft, F(1, 
6)=4.23, p=.085, suggests a tendency for the normal achievers to produce 
more in terms of final drafts on the word processor and less when writing in 
the pencil-and-paper sessions, whereas the learning disabled seem to produce 
more on a final draft using pencil-and-paper. 


Table 1 
Means and Standard Deviations (in brackets) for 
Number of Drafts, Number of Words, and Number of Words in First and 
Final Drafts for Word Processing and Pencil-and-Paper Sessions for 
Learning Disabled and Normal Achievers 


Learning Disabled Normal Achievers 
N=4 N=4 
Number of Drafts 
W-P 3.25 (0.13) 2.25 (0.48) 
P&P 2.10 (0.35) 1.55 (0.68) 
Number of Words 
W-P 1687.00 (598.91) 3102.75 (592.16) 
P&P 1338.00 (195.77) 1488.75 (1262.36) 
First Draft Length 
W-P 95.30 (12.45) 121.63 (65.28) 
P&P 109.80 (18.92) 143.33 (49.71) 
Final Draft Length 
W-P 94.70 (11.98) 132.90 (39.07) 
P&P 121.15 (33.02) 131.93 (38.27) 


a ee ng ee 
Note: W-P = Word Processing and P&P = Pencil-and-Paper. 
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Lexical Revisions 

A four-way (2 x 2 x 3 x 3) ANOVA with Group (learning disabled, normal 
achieving), Setting (word processing, pencil-and-paper), Level of Change 
(First Order, Second Order, Third Order), and Revision Type (addition, 
deletion, substitution) as the independent variables was computed. The final 
three variables were within-subjects variables treated as repeated measures. 
The only effect was a main effect for Level of Change, F(2, 12)=10.23, p<.01. 
As the complexity of the Lexical revision type increases, the incidence of 
revisions decreases. First Order Revisions were most frequent (mean=0.79); 
Second Order Revisions were less frequent (mean=0.48); and Third Order 
Revisions were least frequent (mean=0.187). Means and standard deviations 
are presented in Table 2. 


Infralexical Revisions 


At the Infralexical level a four-way (2 x 2 x 3 x 2) ANOVA (Group by Setting 
by Level of Change by Revision Type) was computed. There were main 
effects for Group, F(1, 6)=13.31, p<.01; for Level of Change, F(2, 
12)=51.57, p<.0001; and Revision Type, F(1, 6)=16.48, p<.01. Two-way 
interactions were evident for Group and Level of Change, F(2, 12)=26.77, 
p<.0001 and Level of Change by Revision Type, F(2, 12)=12.22, p<.01. Also, 
there was a three-way interaction for Group by Level of Change by Revision 
Type, F(2, 12)=3.84, p<.05. To clarify these effects each Level of Change 


Table 2 
Lexical Revisions with Word Processing and Pencil-and-Paper for 
Learning Disabled (LD) and Normal Achievers (NA) 


Word Processing Pencil-and-Paper 
LD NA LD NA 
First Order 
Additions M 1.0 1.5 0.5 0.8 
SD 0.9 3.0 0.6 1.5 
Deletions M 1.0 0.8 0.8 0.5 
SD 0.0 1.0 0.5 1.0 
Substitutions M 0.0 1.8 0.3 0.8 
SD 0.0 werd 0.5 1.5 
Second Order 
Additions M 0.5 1.8 0.3 0.3 
SD 0.5 1.3 0.5 0.5 
Deletions M 0.3 0.8 0.3 0.8 
SD 0.5 1.5 0.5 to 
Substitutions M 0.0 0.5 0.3 0.3 
SD 0.5 0.6 0.5 0.5 
Third Order 
Additions M 0.0 0.5 0.3 0.0 
SD 0.0 1.0 0.5 0.0 
Deletions M 0.0 0.0 0.0 0.0 
SD 0.0 0.0 0.0 0.0 
Substitutions M 0.0 0.8 0.3 0.5 
SD 0.0 1.5 0.5 1.0 
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was analyzed and reported separately. The descriptive data on these separate 
effects are presented in Table 3. 

First Order Infralexical revisions. A (2 x 2 x 2) ANOVA (Group by Setting 
by Revision Type) on the First Order revisions (spelling and punctuation) 
revealed a main effect for Group, F(1, 6)=24.60, p<.003, with the learning 
disabled making more revisions (mean=7.5) than the normal achievers 
(mean=1.88). Also, there was a main effect for Revision Type, F(1, 6)=14.11, 
p<.01, with the spelling revisions being more frequent (mean=8.06) than 
punctuation revisions (mean=1.81). 

Second Order Infralexical revisions. A similar three-way (2 x 2 x 2) 
ANOVA (Group by Setting by Revision Type) on the Second-Order revisions 
(tense/number, abbreviation) revealed no main effects or interaction effects. 

Third Order Infralexical revisions. A (2 x 2 x 2) ANOVA (Group by 
Setting by Revision Type) on the Third Order revisions (paragraphing and 
formatting) revealed a main effect for Setting, F(1, 6)=11.71, p<.02 anda 
Setting by Group interaction, F(1, 6)=7.84, p<.05 which shows that the 
higher incidence of Third Order changes in the word processing setting was 
due to the normal achievers. 


Test of Written Language Data 


Standard scores. A four-way (2 x 2 x 2 x 3) ANOVA on the TOWL 
standard scores (reported in Table 4) with Group (learning disabled, normal 
achieving); Setting (word processing, pencil-and-paper); Test Time (pretest, 
posttest); and Subtest (Vocabulary, Thematic Maturity, Word Usage) as the 
independent variables, with the last three variables being within-subjects 
variables treated as repeated measures, revealed no significant effects for 


Table 3 
Infralexical Revisions with Word Processing and Pencil-and-Paper for 
Learning Disabled (LD) and Normal Achievers (NA) 


Word Processing Pencil-and-Paper 
LD NA LD NA 
First Order 
Spelling M 9.75 5.75 15.75 1.00 
SD 4.50 7.60 9.90 1.40 
Punctuation M 1.80 0.30 2.80 0.50 
SD 2.90 0.50 2.40 1.00 
Second Order 
Tense, etc M 0.00 0.30 0.80 0.80 
SD 0.00 0.50 1.00 1.00 
Abbreviations M 0.00 0.00 0.00 0.00 
SD 0.00 0.00 0.00 0.00 
Third Order 
Paragraphing M 0.50 2.30 1.00 0.00 
SD 0.60 1.90 0.80 0.00 
Formatting M 1.00 2.80 0.00 0.00 
SD 1.20 2.10 0.00 0.00 
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Table 4 
Pretest and Posttest Standard Scores on the Test of Written Language 
(TOWL) for Word Processing and Pencil-and-Paper Conditions for 
Learning Disabled (LD) and Normal Achievers (NA) 


Word Processing Pencil-and-Paper 
LD NA LD NA 
Pretest 

Vocabulary M 6.8 9.3 8.3 10.3 
SD 1.0 3.0 1.0 2.8 

Thematic Maturity M 8.8 : 10.8 9.3 10.5 

SD 1.7 1.7 2.1 1.3 

Word Usage M 8.0 12.8 7.5 12.5 
SD 0.8 ez, 1.7 137 

Word Count M 106.8 104.5 113.3 150.3 
SD 48.6 49.1 40.2 74.5 

Posttest 

Vocabulary M (pss) 10.3 To 9.0 

SD 0.6 2.2 1.9 4.2 

Thematic Maturity M 8.8 10.0 9.0 10.5 

SD 0.5 Pin 1.6 2.1 

Word Usage M 8.8 13.8 8.5 13.3 
SD eal els 1.0 157, 

Word Count M 106.5 128.3 129.5 99.5 
SD 24.1 54.5 2r0 52.4 


Note: The standard scores have a mean of 10 and a standard deviation of 3. 


Setting or Test Time. A main effect for Group, F(1, 6)=9.41, p<.05, was due 
to higher performance for the normal achievers (mean=11.06) compared to 
the learning disabled (mean=8.2). A main effect for Subtest, F(1, 12)=9.17, 
p<.01, and a Subtest by Group interaction, F(1, 12)=7.05, p<.01, showed the 
normal achievers with even higher performance on the Word Usage Subtest. 
Post hoc simple effects tests revealed that normal achievers performed high- 
er on Word Usage in both Settings and at pretest and posttest levels (p<.01). 

TOWL story lengths. A three-way (2 x 2 x 2) ANOVA (Group by Setting by 
Test Time) for the number of words written on the TOWL pretests and 
posttests was nonsignificant indicating both groups were writing stories of 


comparable length both before and after the word processing and pencil-and- 
paper sessions. 


Summary 


Table 5 presents a summary of points where the learning disabled benefit 


from the use of a word processor, and comparisons are included from Morton 
et al. (1989). 


Discussion 
The prediction that the learning disabled students in this study would pro- 
duce more with the use of a word processor was partially supported. When 
students were assigned to the word processing group they produced more 
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Table 5 
Comparisons Between Word Processing and Pencil-and-Paper for 
Learning Disabled (LD) and Normal Achieving Students (NA) 
in the Present Study and in Morton et al. (1989) 


Dependent Present Study Morton et al. (1989) 
Variables LD NA Grade 5 Grade6 Grade7 Grade 8 
Productivity 
# of Drafts S S NS S S S) 
# of Words S S NS S S) S 
First & Final 
Draft Lengths NS NS NS NS S) S) 
Lexical Revisions 
1st Order NS NS NS S NS NS 
2nd Order NS NS NS NS NS S 
3rd Order NS NS NS NS NS NS 
Infralexical Revisions 
1st Order NS NS NS S S S 
2nd Order NS NS NS NS NS NS 
3rd Order NS S S S) S S 


Note: S = Significant Difference in favor of Word-Processing; NS = Nonsignificant Difference. 


draft copies of revised texts and their total output was higher. However, the 
word processing did not lead to increased lengths for first and final drafts nor 
increased revisions. 

Furthermore, there were no differences between word processing and 
paper-and-pencil for (1) pretest-posttest results on the TOWL nor (2) pre- 
test-posttest story lengths on the TOWL. 

The learning disabled were able to use the word processor in the lab 
setting in a manner generally comparable to normal achievers. They showed 
statistically comparable performance to the normal achievers on (1) the 
lengths of first and final drafts, (2) total output, (3) story lengths on the 
TOWL pretests and posttests, and (4) meaning-preserving and meaning-al- 
tering changes. They also showed comparable performance on revisions 
related to tense/number and abbreviations. 

However, the normal achievers did show superior performance on Third 
Order Infralexical revisions (paragraphing and formatting) during word 
processing sessions (see Table 5). This may indicate a higher proficiency level 
on the word processor for normal achievers. This effect is also evident when 
the findings are compared with Morton et al. (1989) in Table 5. As may be 
seen in Table 5 the learning disabled differ from their controls and from the 
comparable grade 6 students in the previous study in the area of revisions. 
However, even the normal achieving controls in the present study differ from 
the grade 6 students in the previous study on first order infralexical revisions 
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and first order lexical revisions. It may be that these effects are masked for 
the controls in the present study by the small sample size. 

In addition, the borderline interaction effects seem to indicate that nor- 
mal achievers may (1) produce more in the word processing setting with 
respect to final drafts (p=.085), and (2) produce more in terms of overall 
output on the word processor (p =.056). The fact that the interactions are not 
statistically significant may also be an artifact of the small sample size. 

The small sample size—often a consequence of research where distribu- 
tion of computers is sparse (Maddux, 1985)—suggests interpretive caution is 
warranted. Yet Neale and Liebert (1973) present a rationale for the use of 
small samples by stressing that, “one ought to place at least as much con- 
fidence in results obtained from a small as from a large number of subjects” 
(p. 32) if only because a much larger mean difference is necessary to detect a 
significant difference. Large mean differences may indicate large practical 
effects. 

Several additional effects were evident. First, the normal achievers had 
an overall lower incidence of First Order revisions (spelling and punctua- 
tion). This would be due to their fewer spelling and punctuation errors when 
compared to the learning disabled. Second, the learning disabled produced 
more drafts than the normal achievers. We suspect this is due to the teacher’s 
emphasis in the learning disabilities class rather than shorter story lengths 
because there were no differences in story length. And finally, test scores on 
the TOWL favored the normal achievers as expected. Their even higher 
performance on the Word Usage subtest, as indicated by the interaction, 
suggests that syntactical processing may be a particularly important dif- 
ference between the learning disabled and the normal achieving. 

The most obvious general conclusion to emerge is that both groups are 
able to utilize the word processor in the lab setting to produce more drafts 
and more words overall. However, the normal achievers seem to be utilizing 
this technology more effectively in some areas (i.e., Third Order infralexical 
revisions and overall output). 

One potentially important issue is the distance between students and 
teachers evident in this lab setting. Morocco and Neuman (1986) have sug- 
gested that the main advantage of word processing is the access the teacher 
has to the process the child uses in writing. The text is more public allowing 
for a more interactive process with other students and the teacher. Ideally, a 
teacher-run lab would alleviate monitoring problems and allow for effective 
interaction and conferencing. Perhaps computer labs could be attached to 
library settings (with a computer-literate librarian) or remedial and special 
education settings. Such a prospect would be amenable to research in most 
school settings and should be addressed in future research, especially with 
children who require additional attention. 
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Exploring the Work of 
School Chiefs: The Case of the 
Ontario Director of Education 


The way in which Ontario Directors of Education understand and conduct their work is 
explored through a reanalysis of interview transcripts and previous studies. Directors 
appear to perceive three distinct sectors within their work environment: board, system, and 
community. Work within the board sector revolves around meetings with the board chatr- 
person, the corporate board, and individual trustees. System sector work centers on manag- 
ing the administrative workflow through meetings with other central office administrators 
and school principals. Attention to public relations, interagency liaison, and the profes- 
sional community emerged as important elements within the community sector. While the 
work of Ontario directors appears similar to that described in previous studies of chief 
executives, they appear to spend more time in scheduled meetings than their counterparts in 
other jurisdictions and their work activities display less of the brevity and fragmentation 
often found to be characteristic of managerial work. The complexity of the administrative 
environment and nature of the work done also appeared to be related to the size of the system 
being administered. ; 


The study of chief administrators is a time-honored way of seeking insight 
into the mysteries of organizations and their administration. In modern 
times Barnard’s (1938) classic dissertation on the Functions of the Executive 
is an exemplar of this approach, as is Mintzberg’s (1973) pathbreaking 
observational study of The Nature of Managerial Work and Hodgkinson’s 
(1978) philosophical treatment of administration. Each of these important 
works adopts a different approach and each offers us different, although 
complementary, insights into the character, work, and purposes of adminis- 
tration and administrators. Each, nonetheless, builds on the common as- 
sumption that studies of roles, positions, and people at the apex of 
organizational hierarchies provide particularly powerful paths toward im- 
proved practical and theoretical understandings of administration. 

This article shares that assumption. My main objective is to discuss the 
work of Ontario directors of education with a view to advancing our under- 
standing of school system administration. The paper is based on data gather- 
ed during a recently completed study of Ontario supervisory officers (Fullan 
et al., 1987). This policy-oriented study investigated the roles and respon- 
sibilities of the full range of central office administrative positions in Ontario 
school systems in order to recommend changes to the province’s supervisory 
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officer certification process. Although the final report of this parent study 
described and discussed the role and activities of Ontario directors, its overall 
objectives and purpose precluded a tightly focused analysis of a single ad- 
ministrative position. The principal investigators provided access to the 
original data on directors to facilitate such an analysis, additional details of 
which are contained in Allison and Allison (1988). The account of the work of 
Ontario directors that is presented here draws directly on the insights gained 
during that reanalysis and on those offered in McLeod’s (1984) earlier obser- 
vational study of 16 Ontario directors. 


Setting 

Ontario is the largest, richest, and most populous of the Canadian provinces. 
Despite the vastness of its total area of over 1 million square kilometers, the 
great bulk of the current population of more than 9 million is concentrated 
along and around an 800-kilometer-long axis extending from Ottawa, the 
national capital, to Windsor at the southwestern tip of Lake Erie. This rich 
agricultural, commercial, and industrial corridor is served by schools operat- 
ed by large-area, county, and city school boards. Geographically this region 
has the aspect of a great arrowhead thrusting some 500 miles below the 49th 
parallel, which forms the boundary between western Canada and the United 
States. Settlement in the middle north of the province follows a rough arc 
from Ottawa, through Sudbury, then along the north shore of Lake Superior 
to Thunder Bay, some 1,400 kilometers distant. Provincial schools in this 
region are administered by large-area district boards, which service networks 
of widely separated small communities. With the exception of a few isolated 
settlements served by small school jurisdictions, the far north is virtually 
unsettled. 

Whereas the formative history and current character of the American 
superintendency is characterized by ambiguity and conflict (Blumberg, 1985; 
Tyack & Hansot, 1982), the Ontario directorship evolved from a much more 
stable tradition of school administration (Allison & Wells, 1989). The British- 
derived political and administrative institutions established in the formative 
decades of this originally Loyalist province fostered an acceptance of central- 
ized authority that was contrary to the “Town Meeting” rooted notions of 
local political autonomy which predominated in the “older colonies” to the 
south. Key legislation enacted in 1850 and 1871 laid the foundations for the 
development of a highly centralized system of school administration which 
endured until the 1960s, vestiges of which still endure. For almost a century, 
town and country schools were administered and supervised by a corps of 
centrally regulated inspectors. Chief administrators of city school systems 
were directly employed by local boards, but their appointment was contin- 
gent on certification and approval by the Ministry. All school system ad- 
ministrators became locally employed with the establishment of the current 
large-area school boards in 1969, but the practice of centralized certification 
examinations remains in effect and all supervisory officer appointments 
must still be approved by the central authority. 

Under current legislation all Ontario school boards with a total enroll- 
ment of 2,000 or more students are required to appoint a director of educa- 
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tion to serve as chief education and chief executive officer (Ontario, 1980, S. 
253). In practice all of the 112 boards! enrolling more than 1,200 students 
have a director, as do three boards enrolling fewer students. The Education 
Act further specifies that directors shall, “within policies established by the 
board, develop and maintain an effective organization and the programs 
required to implement such policies” (S. 253[2]). System job descriptions 
typically specify that directors will advise the board on policy matters, ensure 
the effective administration of the system, and provide leadership. It is also 
common practice for many Ontario boards to assign the additional legal title 
and duties of Secretary to the director’s position. This additional function 
gives directors legal responsibility for board minutes and typically gives them 
charge of preparing the board agenda. 


The Data 


The original Fullan et al. (1987) study was based on data gathered from a 
sample of Ontario school systems drawn so as to ensure appropriate repre- 
sentation on the basis of size (enrollment), geographic location, and constitu- 
tion (public/separate, anglophone/francophone). In each of the selected 
systems, interviews were held during 1985-1986 with the director, the chair- 
person of the board, and some or all central office supervisory officers (SOs), 
depending on total numbers. Interviews lasted about one and a half hours 
and were based on standardized protocols. Directors were interviewed ac- 
cording to the standard supervisory officer schedule (pp. 207-209). Career 
trajectory and demographic data were collected through a short pre-inter- 
view questionnaire. The discussion presented here is based on coded ques- 
tionnaire and interview data from 22 directors supplemented by a reanalysis 
of 18 of the original interview transcripts. 

Ontario school systems vary widely in size, those employing directors 
ranging in enrollment from approximately 1,000 students to over 100,000, 
the median enrollment being 8,700. Such large size variations obviously 
reflect proportional variations in the complexity of internal organizational 
structures and other related factors which could be expected to be associated 
with differences in the nature of the work environment and activities of 
directors. In order to explore this possibility during the reanalysis, the sample 
school systems were grouped into three size categories: Small (<10,000 
students), Large (10,000-50,000), and Very Large (>50,000). Using this 
classification, 7 of the 22 directors discussed here administered small school 
systems (SSS), 11 large school systems (LSS), and 4 very large school systems 
(VLSS).” This “sample” provides an inverse proportional representation of 
the population of Ontario school chiefs, accounting for 48% of the VLSS 
directors (V=7), 16% of those in LSS (V=50), and 12% in SSS (N=58). The 
distribution of the respondents across these categories did, however, allow for 
within-group patterns of responses to be considered and for comparisons to 
be made across the overall size range. 

All but 2 of the 22 directors considered here were men.° All had been 
teachers and most had spent at least several years as principals. Most had 
had considerable central office administrative experience in positions other 
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than that of director, the total sample reporting a median of 7.2 years of 
experience in such positions. Only two members of the sample had served as 
chief school officers (CSOs) of other systems prior to assuming their current 
post, which is consistent with McLeod’s (1984) identification of an Ontario 
tradition discouraging lateral mobility among directors. Most of the CSOs 
considered here, then, were well experienced and seasoned educational ad- 
ministrators who had likely attained their professional apogee. 


Analysis and Limitations 


The transcript reanalysis sought to identify commonalities, contrasts, and 
conceptual themes in the way the directors described and discussed their 
work and, where possible, to relate these to the organizational size of the 
school systems. Particular attention was paid to the responses given to 
interview questions which asked directors to identify the most important 
aspects of their job, the major responsibilities and tasks involved, the people 
and groups dealt with, and the relative influence of these others on the way 
in which directors structured their work. Comments dealing with these and 
related matters were noted, compared, counted, grouped, and regrouped 
using the guidelines for the interpretation of qualitative data discussed by 
Miles and Huberman (1984). 

Some of the interview transcripts were far more detailed than others, and 
this unevenness in the data could have distorted the findings in some re- 
spects. This limitation was borne in mind during the analysis and attempts 
were made to cross-check the conceptual “patterns, themes or ‘Gestalts’” 
(Miles & Huberman, 1984, p. 216) that emerged as the work progressed. 
There were limited opportunities for triangulation within the data, however, 
and it has not been possible to test the representativeness of emergent 
conceptual patterns through further interviews designed to feedback themes 
and images to the directors concerned or to others. Nor has it yet been 
possible to gather additional data in an attempt to validate, refine, or falsify 
the impressions summarized below. Hopefully this article will contribute to 
these additional steps in the research process by communicating the tenta- 
tive insights gained and encouraging further inquiry into the character of the 
work environment of chief school officers. Regardless, the limitations noted, 
together with the relatively small size of the data set examined, should be 
borne in mind when considering the findings reported below. 


Work Patterns 

The image of the director’s work life which emerged from the reanalysis of 
the Fullan et al. (1987) interview data was generally similar to that present- 
ed in Mintzberg’s (1973) influential study of executive work and reaffirmed 
in subsequent studies of school chiefs in other jurisdictions (e.g., Duignan, 
1980; Pitner & Ogawa, 1981; Willower & Fraser, 1979). On the whole the 
directors reported a work week of between 40 and 80 hours, with most of this 
time being spent in interaction with others. 

The transcripts contained references to tasks and duties which embodied 
or reflected aspects of Mintzberg’s (1973) 10 interrelated managerial work 
roles, namely the three interpersonal roles of figurehead, leader, and liaison; 
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the three informational roles of monitor, disseminator, and spokesperson; 
and the four decisional roles of entrepreneur, disturbance handler, resource 
allocator, and negotiator (pp. 167-170). But while there was some evidence in 
the transcripts of the “brevity, variety and fragmentation” that Mintzberg 
(1973, p. 82) found to be so characteristic of managerial work, these qualities 
did not appear to dominate the work of Ontario school chiefs to the same 
degree as in the case of the executives studied by Mintzberg, or the school 
superintendents studied by Pitner and Ogawa (1981) and Duignan (1980). 
Similarly, while it was clear that the work of these Ontario directors was, as 
in the case of other studies, primarily concerned with communication of 
various kinds, and that the basic “tools” through which they conducted their 
work were the five basic media identified by Mintzberg (mail, telephone, 
unscheduled and scheduled meetings, and tours), Ontario school chiefs ap- 
pear to spend more of their time in regularly scheduled meetings than the 
subjects of previous studies. Duignan (1980), for example, reported that his 
eight Alberta superintendents spent roughly a quarter of their time in un- 
scheduled, and a quarter in scheduled meetings, while Pitner and Ogawa’s 
(1981) six California superintendents spent aggregates of 9% and 46% of 
their time in unscheduled and scheduled meetings respectively. McLeod’s 
(1984) published report of his study of Ontario directors, however, has them 
spending one half to two thirds of their working week in “protracted meet- 
ings, usually involving large numbers of participants” (p. 175). 

Timed observational data were not gathered in the original Fullan et al. 
(1987) study, but the interview data support, and in some ways extend, this 
finding. Directors typically reported that they had regularly scheduled ad- 
ministrative committee meetings several times a week as well as frequent, 
often daily, meetings with colleagues and in some cases trustees, plus, of 
course, regular board and board committee meetings. Ontario boards com- 
monly schedule one evening a week for regular meetings during the academic 
year, alternating standing committee and full board meetings every other 
week. Most of the directors, nonetheless, reported blocking out two or more 
evenings a week for meetings, with one sometimes being reserved for public 
relations functions in the community. Whether a schedule such as this results 
in Ontario school chiefs typically spending a greater proportion of their time 
in scheduled meetings than their counterparts in other jurisdictions is not, 
however, entirely clear. Mintzberg (1973), for instance, reported that the 
single school superintendent in his study spent 75% of his time in scheduled 
meetings. 

Even so, attendance at a large number of formal and semiformal 
scheduled meetings will logically reduce the overall amount of brevity and 
fragmentation in the work of directors, and this might well explain why these 
commonly noted characteristics of managerial work were not readily ap- 
parent in the interview transcripts, especially as a fair proportion of the 
scheduled meetings attended by directors will be board or committee meet- 
ings of record wherein business will be conducted with appropriate structure 
and formality. Regardless, being at and in a multitude of timetabled meet- 
ings is clearly the staple of a director’s working life. 
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Work Patterns and System Size 


Mintzberg (1973) also noted that chief executives of larger organizations 
typically attend slightly more scheduled meetings, and that these generally 
last longer and involve more participants than those in smaller organiza- 
tions. The transcript reanalysis found a similar pattern in the case of Ontario 
CSOs, particularly with regard to evening meetings. Whereas most of the 
SSS directors typically reported that they were engaged in meetings two to 
three nights a week, all three VLSS directors declared that they frequently 
attended meetings three to four evenings a week, one noting that during busy 
weeks he would have meetings every night of the week. Similarly, while some 
directors of small and large systems explained that they tried to keep their 
weekends free, others, and most particularly those in larger systems, report- 
ed that they often attended scheduled meetings during weekends.” One of the 
VLSS chiefs, for instance, explained that he had regularly scheduled meet- 
ings on two Saturdays and one Sunday each month. Some of the directors 
who declared that they attempted to keep their weekends free also noted that 
they would often spend part of this “free time” engaged in paperwork, while 
one of the VLSS chiefs specifically explained that he always reserved Sunday 
afternoon for “homework.” Others made similar references to Sunday as a 
kind of “sanctuary day” free of formal commitments which allowed them to 
catch up with outstanding work and reading. 


Work Environment 
Analysis of the transcripts suggested that the directors tended to sector their 
work environment and their responsibilities into three domains: board, sys- 
tem, and community. Each of these sectors appeared to represent a different 
but reciprocal action arena with a distinct set of obligations and objectives. 


Orientation Toward Sectors 


Different directors reported spending more or less time and care on one or 
another sector. In several cases directors understood that the board expected 
them to tighten up certain aspects of system operations and as a consequence 
of this “commission” from the board they deliberately involved themselves 
more in the workings of the system. Others believed that it was important to 
better represent the system to the community and thus they tended to place 
more emphasis on this sector, taking care to develop good links to community 
groups. In another case a director confided that he gave hardly any attention 
to the community sector except for necessary ceremonial events, preferring 
to “leave all that to the trustees.” The relative emphasis placed on the three 
sectors also appeared to be related in some respects and in varying degrees to 
the size of the system. Additional details are discussed below, but in general 
activities involving the board and community sectors figured more prom- 
inently in the transcripts from LSS and VLSS directors, while SSS chiefs 
appeared to be more directly concerned with system operations. Yet, regard- 
less of the relative emphasis each CSO placed on one or another of these 
sectors, most seemed to perceive each as a distinct domain. More important- 
ly, they also appeared to share a common understanding that the elements of 
one sector should not, ideally, intrude in an uncontrolled manner on another. 
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Thus, in addition to regarding each sector with a different gaze, they were 
generally concerned to keep them apart. 

This general desire to keep the elements of each sector in their proper 
place was most evident at the interface between the board and the system 
sectors. As expressed by one director, this is essentially a matter of “buffer- 
ing” the system from the trustees. This is not to imply that the directors 
attempted to sabotage or otherwise disrupt the implementation of board 
policies. On the contrary, the transcripts showed directors as having a strong 
sense of loyalty toward official system policy. Indeed, they recognized a clear 
duty to implement properly constituted board directives; but at the same 
time they recognized a collateral duty to help the board make good policy and 
to try and head off decisions and actions, particularly by individual trustees, 
that they believed would unnecessarily endanger the operation and health of 
the system. Consequently, many of the interview transcripts contained an 
implicit, and at times explicit, distinction between the corporate board and 
its constitutionally supreme authority on one hand, and trustees as fallible 
individuals on the other. 


The Board Sector 


The interview data did not directly illuminate the way in which directors 
interacted with the board in its corporate manifestation, but from clues in 
these data and McLeod’s account it would seem that if things are going well 
the director’s main task during formal meetings will be to ensure that all the 
necessary props and actors are at hand and hope that the talk and action does 
not diverge too far from the roughly drafted script. Control of board meet- 
ings, of course, resides with the chairperson, but the transcripts suggest that 
there will likely be a close working relationship between this key individual 
and the director. 

Directors and chairpeople. Many CSOs commented on the importance of 
this relationship and their frequent meetings with the chairperson to discuss 
the general state of affairs and the business of the next board meeting. Often, 
it appears, meetings between the chairperson and director revolve around 
the agenda for the next board meeting during which both content and 
procedure are reviewed, and potential problems and possible solutions are 
discussed. Semiformal review meetings of this kind may occur weekly, but in 
some instances directors and chairpeople meet much more regularly. One of 
the VLSS chiefs, for instance, reported that he met with his chairperson 
every day, while five of the other directors also mentioned that they met or 
talked to their chairperson “most days.” 

Trustees. Many directors, and particularly those of large and very large 
systems, also reported a great deal of interaction with other trustees. To 
some degree this may have been an outcome of the timing of the research and 
the phrasing of key questions in the interview protocol. Most of the inter- 
views were conducted shortly after school board elections, and many direc- 
tors commented on the need to socialize the new trustees to their role—a 
revealing comment in itself. Unfortunately, interview questions designed to 
probe responsibilities and role also asked respondents to frame their answers 
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with regard to their work “over the last two months.” Nevertheless, it would 
seem that many of the directors, particularly those in larger systems, met 
frequently with trustees to answer questions regarding system operations 
and discuss Ministry policy, community developments, and, of course, board 
business and political issues. This aspect of a director’s work appears to be 
less dominant in smaller systems, trustees being identified far less frequently 
in the SSS transcripts when the respondents were asked to discuss the extent 
to which they worked with different individuals and groups. Even so, in- 
dividual and small group meetings with trustees are clearly a delicate aspect 
of a CSO’s work, for he or she cannot afford to be suspected of taking sides in 
political struggles between trustees. One of the VLSS interviewees was par- 
ticularly candid on this point and took care to point out that he was 
scrupulously open and above board in all such encounters. 

Strong collaborative data on the importance of the relationship between 
the director and board members emerged in the answers trustees gave in 
response to being asked their expectations of the director. As expressed in the 
Fullan et al. (1987) report, trustees said that they wanted: 


good and complete information from the director, but even more they want advice, 
recommendations, and direction. This expectation seems to be paramount, and was ex- 
pressed by virtually all trustees we interviewed. One trustee suggested a further 
refinement, the director “getting trustees to recognize the implications of their own 
ideas.” In other words the director is to protect trustees from the unanticipated conse- 
quences of their own actions.... [Also] trustees assume the director will inform them of 
the political consequences of the policies they suggest, but it involves more than just 
this. As one trustee put it, “I expect the director to anticipate moves on the part of 
various trustees and others in the system, and have a motion in mind that would satis- 
fy the various elements.” (p. 78) 


Flight plans. Clearly advice and guidance of such magnitude and sen- 
sitivity cannot be given in the fishbowl of a public meeting and must, per- 
force, be sought “backstage,” hence the previous reference to the roughly 
drafted script for formal meetings of the corporate board. This tentative 
outline will not necessarily have been drafted by the director and chairperson 
alone, for each trustee will likely anticipate his or her part and that of others, 
as well as the general ebb and flow of debate. Even when the outcome of a 
vote is unpredictable, the alternatives to be decided will likely have been 
consensually culled to eliminate the unpalatable, perhaps during the com- 
mittee-of-the-whole discussion that many Ontario boards hold in camera 
immediately prior to their public meetings. In such a scenario, the function of 
the CSO is analogous to that of an air traffic controller equipped with an 
intermittently functioning and somewhat unreliable radar set. He has a fair 
idea of the location, route, and probable destination of each trustee, and he 
and the chairperson will likely have drafted rough flight plans and discussed 
possible emergency procedures. But there is always the risk of turbulence, 
especially if unpredictable elements intrude. Moreover, political expediencies 
can result in trustees making sudden in-flight maneuvers. 

Difficulties. The details of managing the board sector can be complex, 
especially given the number of trustees, often 15 or more, that sit on the 
larger Ontario boards and the emergence of the “full-time” trustee (Hickcox, 
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1974). It would also be incorrect to imply that the relationships between 
directors, the corporate board, and individual trustees are always respectful 
and orderly. The interview data suggest that tension and conflict can be 
inescapable aspects of the director’s work, especially when a board is divided 
along political lines or trustees involve themselves with system elements, 
frequent references to both of these contingencies being found in the 
transcripts, particularly those from directors of VLSS. Nevertheless, the 
account given above is believed to approach the truth of the matter, although 
it does not fully apply to all situations. One director of a large system, for 
instance, declared that he did not spend a lot of time with trustees because 
“we do our dealings at board meetings, not behind the scenes.” 


The System Sector 


As would be expected, the directors also reported that they interacted fre- 
quently with their supervisory officers and school-based personnel, this 
being, of course, the prime means by which they manage the system itself. 
Working with and through senior colleagues is obviously another important 
medium through which directors draft the scripts, rehearse key parts, and 
ensure the readiness of essential props in preparation for meetings with the 
corporate board and its committees. McLeod’s (1984) composite vignette 
captured this aspect of work with senior staff well: 


In the conference room Alex and his superintendents work through each agenda item 
for tonight’s meeting.... What will the trustees say to our recommendation on the next 
item? The director draws his aides’ attention to a particular document. At this time it 
is potentially too explosive to take to the board. Reroute it through channels!... The 
director wants a briefing on agenda item #3. He knows that it falls within the Super- 
intendent of Business’s bailiwick. Yet trustees like to quiz their CEO on whim it 
seems. Besides, his subordinate will benefit from this rehearsal. Alex takes his turn at 
giving background information. He explains the budget situation ... [and] ... counsels 
his superintendent on how to present the figures to trustees. After 70 minutes and 
scrutiny of 13 agenda items, the administrators disperse. (p. 181) 


In addition to the obvious similarity to a dress rehearsal, this slice of a 
director’s work life shows clearly how he must meld his private knowledge of 
trustees’ “flight plans” with the administrative workflow of the system, 
while at the same time ensuring all the necessary reports and information 
will be on hand and that his people are well briefed and primed for their 
performances. 

Direct and indirect management. Nevertheless, most of a director’s work 
meetings with his subordinate administrators appear to have a tighter focus 
on the internal operations of the system, and in the smaller systems the 
directors reported that they were of necessity directly involved with one or 
more aspects of system operations. One SSS chief, for example, talked about 
his personal involvement in teacher evaluations, another his work im- 
plementing a new elementary school computer program, and several others 
their responsibilities for contract negotiations with teachers and nonprofes- 
sional staff. As noted previously, directors of smaller systems appear to spend 
less time in scheduled meetings, and from the transcript analysis they tend to 
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manage system affairs primarily through individual contacts with other SOs 
and principals. 

Functional and structural differentiation in larger systems ensures that 
system operations are attended to by subordinates, with the coordination and 
supervision of school and system operations in the larger systems being 
handled by senior SOs, and in the largest by deputy directors. In LSS and 
VLSS much of the director’s system sector work is typically conducted in and 
through regularly scheduled “cabinet” meetings with senior SOs, “council” 
meetings with larger groups of system administrators, often including prin- 
cipals, as well, of course, as through discussions with individual superinten- 
dents or ad hoc groups of colleagues. These various internal meetings are the 
media through which most directors of larger systems appear to manage the 
flow of information, issues, and decisions in the organization, and they also 
provide the main means by which they keep in touch with events and devel- 
opments. Several directors described this aspect of their work as “supervising 
the other SOs,” and it would seem that each meeting or encounter allows the 
director to review the status of various issues, projects, processes, and 
proposals, and, if necessary, give additional instructions or advice. New 
problems or concerns are also identified, categorized, solved, delegated, or 
otherwise assigned a place within the administrative workflow. 

Principals. Three quarters of the directors reported that they also had 
direct dealings with principals. Some CSOs reported that they met regularly 
with specific groups of principals, while others said that they only dealt with 
principals “on demand.” Principals figured more prominently in the SSS 
transcripts, and it appears that directors of smaller systems typically rely on 
and interact more frequently with their principals than is the case in larger 
systems with their greater numbers of central office administrators. One SSS 
director explained that he used his principals as consultants, while two others 
specifically identified principals as the group which had the greatest impact 
on their work. None of the LSS or VLSS directors, however, mentioned 
principals in their responses to this question, all nominating either trustees 
or supervisory officers as the group having the greatest impact on their work. 

School visits. Brief meetings with principals were also mentioned as a 
feature of schools visits, with half of the directors offering specific comments 
on this aspect of their work. Three of the LSS transcripts and two of those 
from VLSS revealed that the directors concerned had attempted to 
deliberately integrate school visits into their regular work activities. One LSS 
chief declared that he visited one school each week, another that he visited 10 
schools a month. Similarly, one VLSS director said “Tuesday is school day,” 
another that he tried for half a day each week in schools. Three of the SSS 
chiefs also specifically mentioned school visits, but in two of these cases the 
directors complained that they had difficulty in finding the time to “get out 
to the schools,” a problem that was clearly exacerbated by the large 
geographical area of some systems. This was also a factor in the third 
instance, but in this exceptional case the director explained that because he 
was new to the system he had made it a priority to personally visit every 
classroom. Thus, while some CSOs of both large and small systems saw 
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school visits as an important aspect of their work, the demands of other 
responsibilities and the greater traveling distances involved appeared to 
make this more difficult in the smaller systems. 

Other specific aspects of the director’s work that appeared in the 
transcripts were the generic administrative tasks of making decisions, solv- 
ing problems, and resolving conflicts. Little specific information was given in 
the transcripts on the kinds of decisions, problems, and conflicts that they 
handled personally, but it would appear that many concern or arise directly 
from managing the administrative workflow, while others are rooted in 
disturbances in the board and community sectors, or are associated with an 
intrusion of one sector into another. 

Difficult relationships. One interesting point here was that when asked to 
identify groups with whom they had particularly difficult relationships, direc- 
tors of small and large systems typically identified two, three, or more such 
groups, whereas the VLSS directors named very few, if any, such groups. One 
VLSS chief said that he had recently had a difficult experience with the 
caretakers’ union, but that that particular problem was not part of his formal 
responsibilities and he had chosen to become personally involved in resolving 
the matter. Another simply declared that he did “not really” have any 
difficult relationships. One SSS chief gave a similar answer, but all others 
and all the LSS chiefs identified groups with whom they had what appeared 
to be continuing difficulties. The most frequently mentioned were teacher 
federations (unions) and other employee unions, some respondents explain- 
ing that contract negotiations and grievances posed particular problems. 


The Community Sector 


All but one of the transcripts contained comments dealing with work and 
responsibilities outside the system and board sectors. Taken together these 
comments defined and encompassed four environmental precincts, as it 
were, each of which can be taken as constituting a subsector of the com- 
munity-oriented activities engaged in by directors. 

Public relations. Nine of the directors made specific reference to “PR 
work” of one kind or another. The LSS directors appeared to be more 
concerned with and active in this area. One declared that he spent “a lot of 
time doing PR work in the community,” another that he “does a lot of 
community PR stuff.” For the most part this would seem to consist of 
maintaining contact with and making presentations to what one described as 
“education related groups and service groups.” One CSO said that he 
regularly made “40-50 presentations a year,” and this may not be atypical for 
directors who place a high priority on this precinct of the community sector. 
Specific references were also made to “ratepayers,” and one to the “com- 
munities,” meaning the different towns and other settlements within the 
boundaries of the system. On this point, the director in question explained 
that “we’re going to nail them for increased taxes so we need their support.” 

There were far fewer references to public relations activities in the SSS 
and VLSS transcripts, and those that were made tended to have a different 
flavor. Two SSS directors, for example, cited their involvement in preparing 
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newsletters as an illustrative instance of their public relations work while 
another confessed that he didn’t like to speak to public groups and thus did 
very little. None of the VLSS chiefs talked specifically about PR work, but 
each made references to the importance of representing the board to the 
community, particularly through attendance at community-oriented school 
activities and symbolic and ceremonial events such as civic functions, din- 
ners, retirements, and funerals. As expressed by one VLSS chief, “senior 
administration has to be seen [emphasis added] to be meeting the needs of the 
community.” 

Interagency liaison. The establishment and maintenance of formal or 
semiformal working contacts with local agencies also emerged as an aspect of 
a director’s community-oriented activities. Here emphasis is placed on the 
exchange of information relevant to the operation of the system and, par- 
ticularly in the smaller systems, the direct management and coordination of 
interorganizational relationships between the system and various com- 
munity agencies. One specific example given was the management of shared 
facilities, a task which has gained increased importance and complexity in 
smaller systems with the advent of government funding for Roman Catholic 
high schools and the entrenchment of French language education rights. 
Other examples mentioned in the transcripts included liaison with social and 
municipal agencies. 

Parents. Perhaps surprisingly, only four of the transcripts made specific 
references to parents, but in each case the relevant comments portrayed 
parents as a distinct element within the external environment. Also in each 
case, the references associated parents with problems of one kind or another. 
One CSO, for example, described his relationship with parents as primarily 
one of handling complaints, another explained that he saw his role toward 
parents as being “available and understanding,” while another declared 
somewhat ruefully that “parents think I can solve all problems.” 

Professional community. Finally, there was clear evidence in the 
transcripts that the directors devoted some of their scarce time to maintain- 
ing contact with their professional colleagues and peers and gathering intel- 
ligence at conferences and other information oases. Few specifics were given, 
but it would seem that attendance at Ministry briefings and participation in 
professional activities provide important vehicles for both gathering and 
exchanging information. No reliable measure on how much time CSOs spend 
on this aspect of their work was available, but one declared that he regularly 
devoted 10-15% of his time to “out-of-county” activities. Some directors 
complained that they were caught in a “Catch 22” in this part of their work, 
for while it was necessary for them to spend time out-of-system to gain the 
up-to-date information demanded by trustees, some of their political masters 
also wanted “their” director to be available to them at all times. 


Discussion 
Three broad and related points emerge from the analysis reported above. 
First, the work and role of Ontario directors as represented in the data 
discussed here broadly conforms to previous analyses of executive work and 
that of school system chiefs in particular. As noted at the outset of the 
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analysis and illustrated throughout the account of the findings, the descrip- 
tions which directors gave of their work reflected the managerial roles iden- 
tified by Mintzberg (1973), as well as the nature and texture of the work done 
by the executives he studied and the school superintendents studied by 
others. Given the differences between the historical, organizational, and 
institutional contexts of Ontario directors and American superintendents 
discussed elsewhere (Allison, 1988; Allison & Allison, 1988), this is itself a 
notable finding. 


Generalism 


Second, while there were clear similarities between the findings of this and 
earlier work, at the same time the description and analysis of the work of 
Ontario directors offered here also helps illustrate other aspects of executive 
work which were not stressed in previous studies. One synthesizing concept 
which helps capture some key elements of the directors’ orientation to their 
work environment is provided by Hodgkinson’s (1978) notion of generalism. 
He has argued that whereas management is essentially a specialism, ad- 
ministration is a generalism. He further refined this point by noting that the 
chief executive is the organization’s “specialist in generalism” (p. 17). As 
such he or she is naturally more concerned with ends than means, with the 
broad coordination of effort rather than the handling of specific functions, 
with the future rather than the present, and with planning rather than 
reacting. 

Such characteristics were threaded through the account of CSOs’ work 
offered above, and seem particularly evident in the ways in which directors 
relate to and integrate elements within the board, system, and community 
sectors of their work environment. Their interactions with each of these 
sectors provide them with unique opportunities to accumulate information 
and develop in their minds a broad but complex image of the system which is 
more detailed and complete than that held by others. Their unique know- 
ledge of each of the three sectors together with their broad general under- 
standing of the organization enables them to explain, interpret, and relate 
each sector to the others. In addition, their role requires them to use their 
unique knowledge to balance, arbitrate, and when possible harmonize many 
diverse and often inherently conflictual expectations and demands. In es- 
sence, they must use their specialized generalist understanding of the or- 
ganization and its operations to guide, integrate, and choose between the 
specialized specialist understandings and interests of others. 

This view is supported by a pertinent finding in the original Fullan et al. 
(1987) report. In the course of analyzing the work of all the supervisory 
officers interviewed, the researchers developed three summative style 
dimensions: system driven versus school driven, reflective versus fire fight- 
ing, and generalist versus specialist (pp. 81-85). As would be expected, direc- 
tors were found to be dominantly system oriented, but they were also 
strongly reflective and overwhelmingly generalist in orientation. Their sub- 


ordinate superintendents, on the other hand, were rated consistently lower 
on all three dimensions. 
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Size 

Finally, while there would appear to be broad similarities in the work en- 
vironment of Ontario directors, a number of size-related differences in both 
the nature of the work and the emphasis placed on particular tasks and 
relationships were noted. Thus, while the construct of a sectored work en- 
vironment appeared to accurately account for the way in which directors in 
small, large, and very large systems structured their activities and under- 
stood their main tasks and responsibilities, differences in the emphasis which 
directors placed on their dealings with each sector and differences in the 
ways in which they interacted with each appeared to be at least partially 
related to the organizational size of the system. Other factors such as the 
administrative style of the CSO concerned, his or her personal values, 
specific directives or policies of the board, and so forth would account for 
some of the differences, but the size of the system and related structural and 
environmental factors seem to offer a more consistent and parsimonious 
explanation for some of the patterned differences noted. The tendency for 
directors of SSS to be involved in managing some discrete operational 
aspects of the system while those in larger systems concentrate on coordinat- 
ing and supervising the work of supervisory officers provides one more 
obvious example. The greater emphasis which many directors of LSS ap- 
peared to place on managing the board sector and participating in public 
relations activities might constitute another, as might the more ceremonial 
involvement of VLSS directors in the community sector and the way in which 
they appear to be able to resist entanglement in difficult relationships with 
groups with which the board and other administrators must deal. 

Spectrum of abstraction. Size-related differences in the work of CSOs can 
be related to the notion of generalism discussed above with the assistance of 
an empirically-based theory advanced by Jaques (1976). His scheme recog- 
nizes the existence of a profound disjunction between the administration of 
what he calls direct command and general command systems. In Jaques’ 
words, the heart of this difference involves 

the step from work and management based upon concrete modes of thought to work 

and management based on abstract modes of thought. No longer can the chief execu- 

tive or manager perceptually encompass the whole of his task, even by imaginal scan- 
ning. He must be able to detach himself sufficiently from the concrete situation to be 


able to gain a rounded perspective and grasp it by means of conceptually formulated 
information. (p. 318) 


Jaques’ theory also contemplates the existence of higher and more abstract 
levels of imaginal complexity in the administration of particularly large 
organizations. His notion of general command systems is quite similar to 
Hodgkinson’s (1978) idea of generalism, and taken together the two views 
suggest a spectrum of conceptual abstraction in the governance of organiza- 
tions ranging from concrete, managerial, and specialist emphases and orien- 
tations in smaller systems to more indirect, abstract, and generalist modes in 
larger systems. Pale 
There are problems in matching Jaques’ (1976) organizational size 
categories to school systems, but the administration of at least the smaller 
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systems in the SSS category used here could be taken as embodying elements 
of his direct command systems while the LSS systems would clearly qualify 
as instances of general command systems. As such, Jaques’ theory and the 
notion of a spectrum of conceptual abstraction offers a basis for interpreting 
and relating the size-related difference noted above. Moreover, the VLSS 
could be recognized as Stratum 4 and 5 organizations in Jaques’ scheme, in 
which case his theory predicts that the directors of these particularly large 
systems would be required to work at even higher levels of abstraction and 
generalization. No clear test of this hypothesis was possible with the data 
considered here, but the findings reported above would appear to provide 
some support for such a view. Nevertheless, the Jaques-Hodgkinson notion 
of a size-related spectrum of conceptual abstraction and generalism in the 
orientation of chief school officers to their work environment would appear 
to offer a potentially profitable avenue for future research, especially as it 
embodies profound implications for the education and selection of school 
chiefs of different-sized systems. 


Conclusion 

This article begins by noting that the study of chief executives has tradition- 
ally offered insightful paths to better understandings of administration and 
administrators, and this also seems a suitable point on which to end. Despite 
the limitations of the data considered in the reanalysis reported here, the 
findings and explanatory images discussed suggest that studies of chief 
school officers can yield particularly powerful insights into the administra- 
tion of school systems. As such, further and more detailed examinations of 
the work and work environment of chief school officers would appear 
worthwhile. Although relatively simple and not particularly novel, the con- 
struct of a sectored work environment developed in this paper appeared to be 
solidly anchored in the data and could well provide a useful framework for 
future work. The possible effect of organizational size and related factors of 
the administration of school systems and how chief school officers orient 
themselves to their work environment also seems worthy of further study, 
and the notion of a spectrum of abstraction and generalism outlined above 
may prove useful in this respect. Yet, regardless of the conceptual frames 
which may guide future work in this area, the important point would seem to 
be that we seek ways to better understand how chief administrators under- 
stand their world and their work. 


Notes 


Excluding the Metropolitan Toronto School Board. 

2. The 18 original interview transcripts on which the bulk of the detailed analysis reported 
here was based were distributed across these size categories as follows: seven from SSS, 
eight from LSS, and three from VLSS. 

3. Male pronouns are used throughout the discussion of the transcript reanalysis in order to 
preserve the confidentiality of the interview data and avoid inadvertently identifying the 
two women CSOs in the sample or attributing quotations to them. 

4. One partial exception here concerns some of the northern SSS where the directors noted 

that regular board meetings were held on Saturdays. 


— 
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The Relationship Between 
Teacher Stress and Perceived 
Coping Effectiveness: Gender 

and Marital Differences 


Elementary school teachers (n=781) were surveyed to examine the relationships between 
sources of work-related stress and types and effectiveness of coping strategies. Multivariate 
analysis of variance revealed gender differences on role stress, supervisory support, and job 
and life dissatisfaction. In all cases males felt more stressed than did females. In addition, 
unmarried teachers felt more role stress and life dissatisfaction compared with married 
teachers. With regard to coping, males felt that avoidance coping was more effective than 
did females, whereas females felt that problem solving was more effective than did males. 
Multisample multivariate regression models (LISREL VI) revealed that teachers who were 
most stressed perceived avoidance strategies to be more effective than those who felt less 
stressed, whereas teachers under less stress perceived problem solving techniques to be 
more effective. However, the relationships between stress and coping differed depending on 
whether male teachers were married, but no such differences emerged for the females. 
Finally, absenteeism was related to stress but not related to the coping factors. 


Teaching has been identified as a particularly stressful occupation (Kyriacou 
& Pratt, 1985) and evidence indicates that there is a correlation between 
perceived teacher stress and the experience of psychological and physical 
symptoms (for reviews see Hiebert & Farber, 1984; Needle, Griffin, 
Svendsen, & Berney, 1980; Turk, Meeks, & Turk, 1982). Surprisingly, re- 
search in the stress and coping literature has relatively ignored gender-re- 
lated differences in the appraisal of stressful events and few studies have 
examined role-related differences (e.g., marital status). Given there are sig- 
nificant differences between the sexes in gender-role development, one would 
expect that men and women would differ in their appraisal of stressful events 
(Bem, 1981). Furthermore, because family life provides its members with 
social support, married teachers, compared with unmarried teachers, may 
cope better with work-related stress (Cobb, 1976; House, 1981). 


Coping Strategies 
While considerable emphasis has been given to investigating the causes and 
consequences of teacher stress, few studies have focused on how teachers 
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cope with work-related stress (e.g., Kyriacou, 1981; Needle, Griffin, & 
Svendsen, 1981; Ratsoy, Sarros, & Aidoo-Taylor, 1986). The importance of 
coping as a mediator between the behavior of the person and environmental 
demands has been emphasized by Lazarus and Folkman (1984). According to 
their theory, psychological stress occurs when an individual appraises a 
situation as threatening and perceives his or her resources for coping with it 
as inadequate. Coping is considered an ongoing process and whether stress, 
measured at any one point in time, causes coping, or whether the converse is 
true, is difficult to determine. Lazarus and Folkman (1984) posit that the 
relationship between stress and coping variables is circular. Further, they 
suggest that coping serves two major functions. Coping strategies may be 
directed at the situation (problem-focused) or they may be directed toward 
managing emotional responses (emotion-focused). In most stressful situa- 
tions both forms of coping, albeit in different proportions, will occur 
(Folkman & Lazarus, 1980). 

Most previous studies of teachers’ coping strategies have focused almost 
exclusively on frequency and type of coping (e.g., Kyriacou, 1981; Needle et 
al., 1981). Although the use of coping strategies is important, relatively few 
studies have addressed the effectiveness of coping efforts to buffer stress and 
minimize its effects. This is an important issue because, although a par- 
ticular coping strategy may be used frequently, it may not be effective in 
reducing stress. Common indicators of successful coping include perceived 
helpfulness or effectiveness, reduction in emotional distress, and reduction in 
problem level (Menaghan, 1983). Furthermore, some theorists regard the 
active efforts to change the cause giving rise to problems (e.g., problem 
solving) as healthy, functional, and effective modes of coping whereas passive 
modes of coping (e.g., avoidance, denial) have been viewed as maladaptive 
and ineffective ways of facing stress (e.g., Haan, 1977). 

Studies that have assessed the perceived helpfulness of coping usage and 
emotional distress levels reveal a complex relationship. Berman and Turk 
(1981) found that individuals with lower life satisfaction and negative mood 
states reported coping strategies such as expressing negative feelings as 
more helpful, compared with individuals with positive mood states. Similar 
results were found by McCubbin and others (Boss, McCubbin, & Lester, 
1979; McCubbin, Dahl, Lester, Benson, & Robertson, 1976). Recently, 
Aldwin and Revenson (1987) found that negotiation (i.e., problem-focused 
coping) was related to decreased symptoms when coping effort was perceived 
as successful. When the coping effort was not perceived as successful, how- 
ever, use of negotiation increased emotional distress. Thus taken together 
these studies provide support for the importance of perceived coping effec- 
tiveness in understanding the adaptiveness of coping strategies. 


Gender and Marital Status 

With regard to gender differences, Russell, Altmaier, and Van Velzen (1987) 
found that male teachers reported greater levels of depersonalization than 
females; however, Pearlin and Schooler (1978) reported that males were 
better able to use coping strategies to reduce the impact of work stressors. 
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Etzion and Pines (1986) found that men report “ignoring the source of 
stress” significantly more than women, who “talk about the source of stress.” 
Hamilton and Fagot (1988) found no gender differences in utilization of 
problem solving behaviors for college students. Finally, the meta-analysis of 
gender differences in occupational stress by Martocchio and O’Leary (1989) 
also found no gender differences in experienced and perceived work stress. 
Thus contradictory evidence exists regarding gender differences in work-re- 
lated stress and coping strategies. 

An explanation for gender differences and coping suggested by Etzion and 
Pines (1986) is that they may be due to stereotypical sex-role differentiation. 
Women are expected to be open and to confide, whereas men are encouraged 
to manage on their own (Greenglass, 1982). However, Hamilton and Fagot’s 
(1988) results would not support this contention. 

The ability to cope effectively with work stress may also be related to 
work-place norms. For example, elementary school teaching is a feminine 
sex-typed occupation (i.e., majority of workers are female). Stereotypical job 
qualities considered especially suited to men such as higher status, power, 
prestige, and salaries, jobs requiring competition, action-oriented, and high 
pressure, and jobs that involve scientific and bureaucratic tasks are notably 
lacking in elementary school teaching (Broverman, Vogel, Broverman, 
Clarkson, & Rosenkrantz, 1972). Consequently, it may be that male teachers 
experience greater stress because, as Bem (1975) postulated, individuals 
experience greater stress when prevailing sex-role expectations are incon- 
gruent with occupational role. 


Stressor Types 


Researchers have identified significant sources of teacher stress including 
classroom management, role conflict, relationships with colleagues, and 
work overload (Friesen & Williams, 1985); political climate, negative public 
support, and role definition (Ruddy, 1984); interaction with administrators 
and student pressures (Bartell, 1984). According to Pettegrew and Wolf 
(1982), previous research on teacher stress viewed the phenomenon from 
three conceptually distinct points: (a) role-related stress associated with the 
degree of congruity between the teacher’s expectations of his or her teaching 
role and the actual teaching experiences, (b) task/event stress stemming 
from a variety of specific tasks teachers perform as part of their teaching 
duties, and (c) environmental stress stemming from the global school en- 
vironment. Thus, in this study, all three stressor domains will be included. 
Specifically, the purposes of our research were threefold. The first aim 
was to investigate whether differences in the degree and type of teacher 
stress and in the perceived effectiveness of different coping strategies were 
related to gender and marital status of the teachers. Second, because stress 
and coping are both multivariate concepts, it was of interest to investigate 
the nature and degree of relationships among teacher stress domains and the 
perceived effectiveness of different coping types. This would seem to be 
important because it would appear doubtful that there is only one effective 
coping strategy for all types of stress. The coping strategies that are per- 
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ceived as most effective are likely to vary with the type of stress and with the 
individuals’ life circumstances (Ilfeld, 1980; Pearlin & Schooler, 1978). As 
the consistency of coping effectiveness with different work-related stressors 
is relatively unexplored, no predictions were made with regard to expected 
results. Finally, because absenteeism has been found to correlate with psy- 
chological and physical distress for teachers (Bridges, 1980; Litt & Turk, 
1985), the association between absenteeism and stress and coping was 
studied. 


Method 
Subjects and Procedures 


The data were collected as part of a large-scale needs assessment conducted 
by the district school board and the city health department (Long, Schutz, 
Kendall, & Hunt, 1986). The target group for this study was defined as the 
population of elementary teachers employed in a metropolitan school district 
in western Canada. The questionnaire was distributed late September and 
collected through the regular school district mail delivery system, with the 
request that it be completed and returned within two weeks. Questionnaires 
were returned by 879 elementary teachers representing a 51% response rate. 
The sample appeared to be generally representative of elementary teachers 
in the district. Demographic characteristics of the sample were 21% males 
and 79% females, median age of 42 years (range 22 to 64), 57% married, 41% 
had children at home, 67% had more than 10 years teaching experience, and 
the median class and school sizes were 21-30 pupils and 300-800 pupils, 
respectively. After discarding questionnaires with missing data, 781 subjects 
were retained for analyses. 


Measures 


The questionnaire had 182 separate items divided into six subsections. Only 
the subsections used in the present study are described in detail. The first 
section contained demographic, occupational, and absenteeism information 
(i.e., total number of sick days taken in the previous year, rated “1-3,” “4-10,” 
“11-20,” and “>20” sick days). 

Stress. Pettegrew and Wolf’s (1982) Teacher Stress Measure was used for 
this study because it was developed from a sound theoretical base and has 
undergone some initial psychometric analyses. Work-related stress is 
theorized to arise from three conceptually different sources, resulting in 
“role-related stress” (how well teachers’ expectations of the teaching role fit 
the actual experience of fulfilling that role), “task/event stress” (tasks 
teachers must perform), and “environmental stress” (experienced at the 
organizational level). In addition, Pettegrew and Wolf included two subscales 
relating to global feelings of well-being at work and surrounding life in 
general. 

Responses to items were on a 5-point Likert-type scale labeled from 
“strongly disagree” to “strongly agree.” Positively phrased items were 
reverse scored for analyses so that a score of 5 represented a high-stressed 
(ie, degree of stress) response for all items. Recent confirmatory factor 
analysis work by Schutz and Long (1988) has reduced this inventory to fi 
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factors comprising 36 items. Cross validation with confirmatory factor 
analysis, reliability analyses, and divergent groups validation indicated that 
this revised inventory has a theoretically verifiable factor structure, good 
internal reliability, and support for construct validity. The factors (i.e., type 
of stress) are labeled as Role Ambiguity (RA, 5 items); Role Stress (RS, 7 
items); Organizational Management (OM, 5 items); Task Stress (TS, 6 
items); Supervisory Support (SS, 4 items); Job Dissatisfaction (JD, 4 items); 
and Life Dissatisfaction (LD, 5 items). The internal consistency coefficients 
(alpha) ranged from .74 for Role Ambiguity to .87 for Role Stress. Subscale 
scores were found by averaging appropriately reflected item scores. 

Coping. Coping refers to any response to a stressor that serves to prevent, 
avoid, or control psychological distress (Pearlin & Schooler, 1978). Coping 
type and effectiveness were assessed by responses to an 11-item scale 
developed for this study. Similar to Lazarus and Folkman’s (1984) Ways of 
Coping Scale format, respondents were asked to identify the most salient 
work-related stress experienced during the past month and to indicate the 
degree of perceived effectiveness for only the coping strategies they used. The 
scale ranges from 1 (ineffective) to 5 (extremely effective). The scale includes 
items representative of coping subscales identified by other researchers (Bill- 
ings & Moos, 1981; Lazarus & Folkman, 1984): seeking information, ignor- 
ing feelings, problem solving, changing the cause, withdrawal, seeking 
support, cognitive reappraisal, and four symptom-management strategies 
(i.e., rest, exercise, recreation, and substance use). In order to best represent 
the two theoretically identified functions of coping, problem-focused and 
emotion-focused, only 7 of the 11 items were used to comprise two subscales 
representing Problem solving (PS) coping and Avoidance (AV) coping. Rest, 
exercise, and recreation were considered ambiguous regarding PS and AV 
functions and were not included. The 7 items are presented in Table 1. A 
confirmatory factor analysis using LISREL VI (Joreskog & Sorbom, 1984) 
supported this two-factor hypothesis. Unweighted least-squares parameter 
estimation based on a polychoric correlation matrix yielded an adjusted 
goodness-of-fit index of .99, and a root mean square residual equal to .04. 
Furthermore, no residual correlations exceeded .10. These indices when 
taken together indicated a good fit of the two-factor model to the data. 


Analyses 


Gender and marital status. The purpose of this set of analyses was to 
investigate whether differences in the degree of stress and in the perceived 
effectiveness of the different coping strategies are associated with gender 
and marital status. The two measures of coping effectiveness were deter- 
mined by the averaged item scores on the Problem Solving and Avoidance 
subscales. Differences among the subgroups were tested using a gender by 
marital status (2 x 2) MANOVA with Problem Solving and Avoidance as the 
dependent variables. For this study, the two levels of marital status were 
single (i.e., unmarried, divorced, separated; n=315) and married (n=442). 
There were few teachers who responded in other ways (e.g., common law, 
n=24). These teachers were not included in the analyses. MANOVAs were 
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Table 1 


Factor Loadings and Items for the Coping Strategies (n = 781) 
Pee aerate Be eh, ot res cones tt yeh bog or com ered ined alt We 


Factor Loading Item 
ce SAI E ANS , FR, Eh ni es eth ee oe 
Problem Solving 53 Seek more information 

.63 Problem solving 

57 Ask others for help 

58 Attempt to change the cause of stress 
Avoidance 53 Act the opposite of the way you feel 

73 Drugs, smoking, or alcohol 

65 Withdrawal from situation 


done separately for the two coping strategies and the seven stressor vari- 
ables. The level of significance for both the multivariate test statistic and the 
follow-up univariate tests was .05. 

Relationship between stress and coping. This set of analyses set out to 
investigate the nature of the relationships between the stressors and the 
coping strategies. Of particular interest were two questions: (a) What is the 
- nature and strength of the relationships among the stressors and the coping 
strategies within the four subgroups used in this study (married and unmar- 
ried females, married and unmarried males)? (b) Are the coping-stress 
relationships similar across the four subgroups? 

These questions were addressed by using multisample multivariate re- 
gression analyses with LISREL VI. In these analyses, each of the seven types 
of stress was regressed on each of the coping strategies. This procedure 
differs from the multiple regression framework where seven independent 
regressions would be calculated. In multivariate regression, regression coef- 
ficients are estimated accounting for the correlations among both the inde- 
pendent and the dependent variables. This procedure yields regression 
coefficients which measure the relationship between two variables after the 
effects of other variables in the equation have been taken into account. The 
standardized scores of each variable were used to control for scale differences 
among the variables and to thereby enable a comparison among the various 
regression coefficients. By constraining the regression coefficients in appro- 
priate ways it was possible to test for equality among coefficients between the 
subgroups. It should be noted that the use of the regression procedures does 
not imply a direction or causal order. The size and direction of the coeffi- 
cients, which indicate the relationship among stressors and coping strategies, 
are of interest. 

Specifically, the two research questions were investigated in the following 
way. In each of the four subgroups, the correlation matrix among all stressors 
and coping strategies can be described by a model which includes, among 
other parameters, regression coefficients indicating the stress-coping strat- 
egy relationships. A chi-square goodness-of-fit testing the hypothesis that the 
same stress-coping regression coefficients are common across all subgroups 
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is possible; however, it is sensitive to sample sizes and thus should be used in 
conjunction with other measures of fit. Therefore, in this study, the validity 
of the hypothesis was assessed by using two criteria—the chi-square good- 
ness-of-fit statistic and the magnitude of the residual correlations. In analyz- 
ing the residuals we adopted McDonald’s (1985) general strategy that, with 
moderate original correlations, a residual correlation between two variables 
greater than +.10 is indicative of the regression model failing to adequately 
explain the relationship between those two variables. If the regression model 
did not adequately explain the relationships among the variables then, using 
the information in the residual correlations as a guide, some equality con- 
straints between coefficients were removed and a revised hypothesis was 
tested. A regression model was assessed to adequately explain the 
stress/coping relationships in the data when there were no residual correla- 
tions greater than +.10. 

Large differences in the sample sizes in the four groups dictated a slight 
modification of the analytic strategy. Because the subgroups relating to 
females were large and relatively equal (n =266 for single females and n=356 
for married females) compared to the subgroups for males (n=49 for single 
males and n=86 for married males), the two female groups and the two male 
groups were compared separately. Analyzing all four groups simultaneously 
would have biased the estimates of the regression coefficients toward the 
female data and would have also resulted in a misleading chi-square good- 
ness-of-fit test. 

Relationship among stress, coping, and absenteeism. We were interested 
in whether there was a relationship between absenteeism and the various 
stressors or coping strategies. Absenteeism was the total number of days 
absent from work in the preceding year. For the purposes of this analysis, 
teachers were classified into one of two groups based on whether they had 
been absent no more than three days. The dichotomization on this variable 
was done to make the sample sizes more equal. Some support for the validity 
of the absenteeism data was provided by an analysis that indicated that a 
greater proportion of individuals with three or less days absent reported their 
health as excellent, and a greater proportion of individuals with greater than 
three days absent reported their health as very poor or below average, X~(3, 
N=799)=32.2, p<.01 (i.e., “Compared to other persons your age, how would 
you rate your health” on a 4-point scale from very poor to excellent?). A 
MANOVA was performed for the stress and coping variables separately. The 
level of significance of both the multivariate test and the univariate follow-up 
ANOVAs was set to be .05. 


Results and Discussion 
Gender and Marital Status 


The results from the MANOVAs using the work-related stressors and the 
coping strategies are summarized in Table 2 which indicates that there are 
significant differences in the degree of perceived stress for both the marital 
status and gender main effects but not for the interaction. The follow-up 
univariate ANOVAs suggest that there are gender differences on role stress, 
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supervisory support, job and life dissatisfaction. In all cases males feel more 
stressed than do the females (see means in Table 3). Similarly, unmarried 
teachers feel more stressed as compared to married teachers on role stress 
and life dissatisfaction. 

With regard to coping, an examination of Table 2 reveals significant 
differences in the perceived effectiveness of the coping strategies for gender, 
but not for marital status or the gender by marital status interaction. Again, 
looking at the univariate F's and the means in Table 3, females indicate that 
problem solving techniques are more effective than do males, whereas males 
indicate that avoidance coping techniques are more effective than do fe- 
males. However, for both males and females, from an examination of the 
means it is apparent that problem solving coping is perceived to be more 
effective than avoidance coping. 

These results tend to support Etzion and Pines’ (1986) finding that men 
and women report different coping strategies corresponding with stereotypi- 
cal sex-role differentiation. The greater perceived efficacy of problem-fo- 
cused coping of female teachers compared to male teachers (i.e., seeking 
information, asking for help, problem solving, and attempting to change the 
stressor) may result in both socioemotional provision of understanding and 
instrumental provision of assistance—the consequence of which is less per- 
ceived stress for the women. However, an alternate explanation for greater 


Table 2 
Multivariate and Univariate Tests of Marital Status and Gender on Stress 
and Coping Strategies (n = 757) 


Status Marital Gender Marital 
Status x Gender 

Multivariate F 5.41 4.07 0.98 
pK 0.01 0.01 0.45 

Univariate* le p< F p< 

Role Ambiguity 0.03 0.86 1.30 0.26 

Role Stress 4.41 0.04 9.83 0.01 

Org. Management Bee, 0.08 1.63 0.21 

Task Stress 0.01 0.92 3.05 0.09 

Supervisory Support 0.58 0.45 8.85 0.01 

Job Dissatisfaction 1.56 0.22 13.02 0.01 

Life Dissatisfaction 25.15 0.01 16.12 0.01 

Se nn es 8 re a ee ee ee eee ot 

Multivariate F 2.31 8.20 er 
jePes 0.10 0.01 0.77 

Univariate* F p< 

Problem Solving 4.29 0.04 

Avoidance 8.66 0.01 


a ere a nr nT ee 


*Univariate tests are inappropriate if nonsignificant multivariate F. 
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Table 3 
Means (Standard Deviations) of Stress and Coping Variables 
Stress* Single Married 
(n=315) (n= 442) 

Male 2.12 (0.83) 2.11 (0.76) 
RA 

Female 2.11 (0.76) 2.03 (0.67) 

Male 2.73 (0.80) 2.54 (0.72) 
RS 

Female 2.46 (0.76) 2.33 (0.79) 

Male 2.34 (0.83) 2.22 (0.72) 
OM 

Female 2.26 (0.68) 2.13 (0.65) 

Male 2.94 (0.77) 3.02 (0.69) 
TS 

Female 2.88 (0.80) 2.82 (0.73) 

Male 2.26 (0.72) 2.22 (0.74) 
Ss 

Female 2.06 (0.73) 1.99 (0.74) 

Male 2.56 (0.87) 2.54 (0.70) 
JD 

Female 2.37 (0.71) 2.22 (0.70) 

Male 2.52 (0.93) 2.10 (0.71) 
LD 

Female 2.17 (0.72) 1.88 (0.69) 
Coping Strategies” 

Male 2.94 (0.84) 2.94 (1.13) 
PS 

Female 3.22 (0.96) 3.07 (1.07) 

Male 1.63 (1.09) 1.42 (1.10) 
AV 

Female 1.35 (0.92) 1.15 (0.95) 


“Higher the scores greater the stress or coping effectiveness. 
Note: RA= Role Ambiguity, RS = Role Stress, OM = Organizational Management, TS = Task 
Stress, SS = Supervisory Support, JD = Job Dissatisfaction, LD = Life Dissatisfaction, 
PS = Problem Solving, AV= Avoidance. 


male stress may be found in Bem’s thesis (Bem, 1975; Page, 1987) that 
greater stress is experienced when prevailing sex-role expectations are incon- 
gruent with occupational role. 


An examination of the items on the supervisory stress scale also impli- 
cates sex-role differentiation. The items reflect inadequate communication 
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with the school principal (e.g., doesn’t listen, doesn’t pay attention), as well 
as lack of support (e.g., doesn’t support teacher in conflicts with parents or 
outsiders). Possibly, male teachers, due to sex-role constraints, do not admit 
they have problems and do not seek support by asking the principal or others 
for help. It appears that sociocultural beliefs defining the appropriateness of 
occupational behavior for men and women need to be examined and con- 
sidered when developing stress-management programs for elementary 
teachers. 

Similar to other studies (Cooke & Rousseau, 1984; Holt, Fine, & Tollef- 
son, 1987), marital status differences revealed that unmarried teachers feel 
more stressed than do married teachers on role stress and life dissatisfaction. 
However, there were no differences in the perceived effectiveness of problem 
solving or avoidance coping strategies for married or unmarried teachers. 
One could speculate that greater role stress experienced by unmarried 
teachers, compared to their married co-workers, is related to the salience of 
organizational demands due to the economic meaning of work-related events 
in their lives (Brief & Atieh, 1987). Other research on stress and social 
support suggests that the support system implied by family roles (e.g., hus- 
band, wife) can moderate the impact of work-related stressors (LaRocco, 
House, & French, 1980). 


Relationship Between Stress and Coping: Females” 


Similar stress-coping relationships (as indicated by the equal regression 
coefficients) were found in both the single-female and married-female groups 
(7(14)= 13.19, p=.51). Furthermore, neither group had any residual correla- 
tions that exceeded +.10. The regression coefficients are displayed in Table 
4. It is clear from this table that there is a consistent inverse relationship 
between problem solving coping and the stressors indicating a general trend 
that greater perceived effectiveness of problem solving coping is related to 
less stress. Interestingly, this relationship is the strongest between problem 
solving and life dissatisfaction. The relationships between avoidance coping 
and the stressors also show a consistent pattern; however, they are in the 
opposite direction. It appears that female elementary teachers who experi- 
ence more stress also find avoidance coping to be effective. It should be 
cautioned at this point that the strength of the relationships are not strong 
(i.e., regression coefficients are all less than .30) and one should be careful 
not to overgeneralize these results. Nonetheless, these coefficients are consis- 
tent and are all significant for both groups (¢-values greater than 2.00 as 
calculated by LISREL VI). 


Relationship Between Stress and Coping: Males 


The results of the analyses comparing the single-male and married-male 
teachers were not as straightforward. The initial chi-square test assessing 
the equality of the regression coefficients between the two groups yielded 
nonsignificant results, indicating that the same model is plausible for both 
groups (47(14)=9.21, p=.82). After examining the residuals, however, it was 
found that there were several important relationships in the single-male 
group that were not adequately explained. For example, the residual correla- 
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Table 4 
Regression Coefficients for Stress and Coping Variables 


Females Males 

Single and Married Single Married 

(n = 266) (n = 356) (n = 49) (n = 86) 
Stress” PS AV PS AV PS AV 
RA -.10 18 -.32 .46 -.28 .46 
RS -.12 25 -.31  .38 -.08 .38 
OM lien Say ~.28 36 an): Co 
TS ~.15 PA -.36 .40 -.19 .40 
Ss -.12 16 -.42  .30 -.29 .30 
JD -.17 15 -.48 .29 -.23 .29 
LD -.27 .22 -.47  .29 -.15  .29 


Acee Table 3. 


tions between problem solving and each of role ambiguity, organizational 
management, job dissatisfaction, and task stress were all greater than +.10. 
Based on these findings it was decided to attempt to fit a new model in which 
the regression coefficients describing the problem solving relationships could 
differ between single and married males. This model again yielded a nonsig- 
nificant test statistic ((7(7)= 1.36, p=.99) but, more important, none of the 
residual correlations in either group was greater than +.10. These regression 
coefficients are presented in Table 4. 

The pattern of results is similar to those of the two female groups. 
Problem solving and avoidance coping are negatively and positively related, 
respectively, to the seven stressor variables. However, there is a large dif- 
ference in the strength of the relationships (i.e., stronger relationships exist 
for the males). If we first consider the avoidance-stress relationships, it is 
clear that there are consistent and fairly strong relationships between 
avoidance and all of the stress variables for both single and married men. It 
is also interesting to note that the strongest relationships are with the 
specific work-related stressors (i.e., role ambiguity, role stress, organization- 
al management, and task stress). 

One interesting difference between single and married male teachers is 
that of the problem solving life dissatisfaction relationship (regression coeffi- 
cient=—.47 and —.15 for single and married males, respectively). While these 
results are interesting, they are based on relatively small samples of single 
(n=49) and married (n=86) males due to the majority of female teachers in 
the elementary school system. It would be desirable to replicate these 
analyses with a larger sample. 
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Females and Males Compared 


A comparison of males and females indicated that despite males finding 
avoidance coping to be more effective than did females and the opposite being 
true for problem solving coping, the relationship between coping and stress 
was generally in the same direction for both groups. The greater the per- 
ceived effectiveness of problem solving coping the less stress. These findings 
may account for some of the inconsistencies in the literature, that is, al- 
though type of stressors and copy strategies may vary by gender (e.g., Etzion 
& Pines, 1986; Russell et al., 1987), the relations of stress and coping 
effectiveness by gender are similar (e.g., Hamilton & Fagot, 1988). 

With regard to gender and marital status, perceived effectiveness of 
problem solving strategies was associated with less stress for all stressors for 
both married and single females and was the strongest for life dissatisfaction 
and the weakest for role ambiguity. A different pattern emerged for males, 
depending on whether they were married or single. The problem-solving- 
stress relationship was stronger for single males, especially for job and life 
dissatisfaction. An examination of the subdomain means and standard devia- 
tions revealed that single men were more variable in their responses than 
were married men, especially in their responses to life dissatisfaction. This 
fact may account for the stronger problem-solving-stress relations for single 
men. In contrast, single and married women responded with greater 
similarity. The consistency of women’s responses may be due to the 
predominance of female teachers in elementary schools and is in keeping 
with Bem’s (1981) thesis that stress and coping is a function of the consisten- 
cy between sex-role expectations and occupational role demands. 


Relationship Among Stress, Coping, and Absenteeism 


It is clear from Table 5 that there is a significant relationship between the 
total number of sick days taken and the stressors. There are differences in 
the two levels of absenteeism on role ambiguity, role stress, task stress, and 
job dissatisfaction. After examining the means in Table 6, it appears that the 
teachers who were absent not more than three days in the past year had 
lower self-reported stress. It should be noted that the differences in the 
means are not large but they are consistent in their direction. There were no 
differences in the perceived effectiveness of the coping strategies between the 
two levels of absenteeism (F(2,778) =2.90, p<.06). 
The finding that absenteeism was not related to scores on the coping 
factors was unexpected because this study and others have found that 
teachers’ reports of absences were significantly correlated with stress, and 
that stress and coping were related (Litt & Turk, 1985). While the differences 
in the perceived effectiveness between the two levels of absenteeism were not 
large enough to be significant (p <.06), an examination of the means revealed 
that greater absenteeism was associated with greater perceived effectiveness 
of avoidance coping and less effectiveness of problem solving coping. It could 
be that the absenteeism measure in this study was not sensitive enough to 
detect these relationships. For example, Kyriacou and Sutcliffe (1979) report 
that teachers consistently underestimate the number of their absences in the 
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Table 5 
Multivariate and Univariate Tests of Differences 
in Absenteeism on Stress and Coping 


Stress Coping 
n=776 n=781 
Multivariate F 3.62 2.90 
p< 01 .06 
Univariate* EB p< 
Role Ambiguity 5.07 03 
Role Stress 9.5857 01 
Org. Management Nee yi .26 
Task Stress 5.19 .03 
Supervisory Support 0.67 42 
Job Dissatisfaction 14.47 01 
Life Dissatisfaction 3.45 06 


*Univariate tests are inappropriate if nonsignificant multivariate F. 


Table 6 
Means (Standard Deviations) of Stress 
and Coping Variables by Absenteeism 


Sick Days 
0-3 >3 
Stress (n = 427) (n = 349) 
Role Ambiguity 2.04 25 
(0.66) (0.73) 
Role Stress 2.35 202 
(0.75) (0.80) 
Organizational Management 2.18 2.23 
(0.67) (0.70) 
Task Stress 2.82 2.96 
(0.77) (0.72) 
Supervisory Support 2.08 2.04 
(0.72) (0.73) 
Job Dissatisfaction 2.24 2.44 
(0.68) (0.79) 
Life Dissatisfaction 2.00 2.11 
(0.73) (0.76) 
Coping Strategies (n = 431) (n = 350) 
Problem Solving 3.13 3.04 
(1.01) (1.05) 
Avoidance 1.23 1.35 
(0.92) (0.95) 


320 


Teachers’ Stress and Coping 


school year. Furthermore, absenteeism may not only be a reflection of the 
negative health consequences of high stress, but may also be a form of coping 
reflecting either function, problem solving, or avoidance. Absenteeism may 
be a way of avoiding stressors or it may facilitate problem resolution, for 
example, by providing time for class preparation. A closer examination of 
absenteeism is necessary in order to determine its role in the stress coping 
process. 


Summary and Conclusions 


A major issue raised by this research concerns the extent to which the type 
and effectiveness of coping strategies were consistent across subgroups and 
stressors. Although the relationships were modest, the most consistent find- 
ing across both gender and marital status and stressors was the positive 
relationship between the perceived effectiveness of avoidance coping and the 
degree of stress for all the stressors. These relations parallel those found by 
others (Berman & Turk, 1981; Boss et al., 1979; McCubbin et al., 1976, 
Pearlin & Schooler, 1978), that is, greater distress was related to greater 
reported coping effectiveness for avoidance strategies (e.g., acting the op- 
posite of the way you feel, using drugs or alcohol, smoking, withdrawing from 
the situation). Furthermore, the finding that greater perceived effectiveness 
of problem-focused strategies is related to less stress is consistent with 
Aldwin and Revenson’s (1987) results that particular problem focused 
strategies were related to decreased symptoms (less stress) when coping 
effort was perceived as successful. 

The greater role stress (j.e., role conflict and overload) reported by males 
and unmarried teachers is a classic source of stress which has been re- 
searched extensively (Ursin & Murison, 1983). Role stress is linked to both 
mental health and physical outcomes; for instance, individuals whose roles 
are in conflict experience more tension and are less satisfied with their jobs 
than are those individuals whose roles do not conflict, a finding that has been 
replicated across a variety of occupational groups (for a review see Van Sell, 
Brief, & Schuler, 1981). The patterning of job and life dissatisfaction and role 
stress found here is consistent with this previous research. However, future 
research using longitudinal designs and path analysis is required to establish 
the direction of causality, as these findings are solely correlational and 
cannot be interpreted in a causal manner. In fact, recent views tend to 
formulate the cause-effect relationships between stress and coping variables 
as circular (Lazarus & Folkman, 1984). However, regardless of cause and 
effect, it appears that a positive relationship between perceived coping ef- 
ficacy and stress might still reflect the maladaptiveness of this behavior, 
whereas a negative relationship might reflect effectiveness. 

A limitation of major concern is the cross-sectional nature of the study; 
the data reflect elementary teachers’ stress at a particular point in time. 
Thus longitudinal studies need to be carried out to determine whether the 
particular stressors and coping strategies described here are stable or simply 
artifacts of the particular time the survey was administered. Furthermore, 
the survey is based entirely on self-report information. 
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Although the relationships reported here are modest, the findings have a 
number of implications for future research. As noted earlier, gender and 
marital status are related to stress and coping strategies, and perceived 
effectiveness of coping is differentially related to degree of stress. It would be 
important to determine whether the impact of coping varies systematically 
at different levels of situational stressfulness (i.e., main effects or buffering 
effects). Research is also necessary to determine the effects sex-role orienta- 
tion, occupational stereotyping, and differential work salience have on teach- 
er stress and coping behaviors. Future research should directly examine 
frequency of coping and its perceived efficacy to test the role of coping in the 
maintenance of maladaptive responses and the specific effect of directly 
altering coping within therapeutic interventions (cf. Long & Haney, 1988; 
Martelli, Auerbach, Alexander, & Mercuri, 1987). 


Notes 


1. Listing of authors was determined by a coin toss as authorship is shared equally. 
2. Correlation matrices are available from the authors, Counselling Psychology Department, 
210 5780 Toronto Road, University of British Columbia, Vancouver, BC V6T 1L2. 
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Career Progress and Satisfaction 
Among Canadian Academics 


Career progress and satisfaction of Canadian academics were investigated. A questionnaire 
was sent to a stratified random sample of academics, numbering 997, with a return rate of 
44.8%. Descriptive data were reported using percentages and comparisons using chi 
squares, While the percentage of academics satisfied with career choice was high (80.7%), 
34.7% of the respondents would consider a career change at this time. Comparisons by 
gender, rank, desire for change, and preferred professional activities were presented, us 
well as perceived positive and negative aspects of working in universities. The results were 
related to Hill’s (1986-1987) expansion of Herzberg’s (1968) two-factor approach to job 
satisfaction and dissatisfaction. 


In the last decade, many descriptive studies of academics have appeared in 
the literature. In particular, these data most frequently have related to 
academics in the United States (e.g., “Who Faculty,” 1985) and often have 
represented specific professional groups (e.g., psychologists in studies such as 
Pyke, 1977; Russo, Olmedo, Stapp, & Fulcher, 1981). 

According to the United States data, gender, rank, highest degree, pro- 
ductivity, years in service, and administrative roles are interrelated (Astin & 
Bayer, 1972). Much of the literature focuses on gender. Several studies have 
reported that women cluster in the lower ranks (Anderson, 1974; Hill, 1984; 
Kimmel et al., 19774; Russo et al., 1981), are underrepresented in faculties of 
universities and in administration (Brooks, 1983; Russo et al., 1981), and are 
less likely than are men to have had or served as mentors (Lincoln, 1986). 
Patterson, Sutton, and Schuttenberg (1987) noted a relationship between 
rank and mobility. Astin (1972) observed that women are more likely than 
men to move from academic to nonacademic settings. Lincoln (1986) at- 
tributed this to social pressures which make it easier for women than for men 
to release their commitments to the professoriate, even after tenure is 
granted. 

Pyke (1977) reviewed both Canadian and United States data in her article 
on female psychologists. She indicated that women tended to have heavier 
teaching loads, more interest in teaching, and smaller classes than did men. 
They also held lower ranks and fewer doctoral degrees than did men. Pyke 
pointed out that discrimination against females may relate to socialization of 
men and women in terms of traditional sex roles. 


Beverly Timmons is an associate professor in the Department of Psychological Foundations, 
Faculty of Education. Currently, she is serving as Associate Dean of Education. Her research 
interests are in the areas of program evaluation, teacher education, and self-concept. 
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Discrimination in hiring more men and paying them more than women 
has been noted by Lincoln (1986) and Tolbert (1986). It would seem that if 
productivity could be shown to be lower for females than males, the hiring 
and salary differences could, at least in part, be justified. However, Tolbert 
indicated that although claims have been made that male academics are 
more productive than are females, productivity differences disappear when 
disciplines and types of institutions are controlled. 

General work satisfaction and its sources have been studied. Hill (1984) 
indicated that there are few differences in job satisfaction related to gender. 
Diener (1984), Pearson and Seiler (1983), and Barrett, Barrett, and Shanker 
(1983) found that involvement in teaching was rated as one of the most 
important sources of satisfaction among academics. Other sources of satis- 
faction were stimulation and personal growth (Diener, 1984), research 
(Pearson & Seiler, 1983), flexibility in schedules and autonomy (Diener, 
1984), and collegial relationships (Barrett et al., 1983). 

The greatest dissatisfaction appeared to be in the younger and lower 
ranks (Hill, 1984) and among the least mobile (Patterson et al., 1987). 
Dissatisfaction with academic work has been attributed to low salaries 
(Diener, 1984; Pearson & Seiler, 1983), working conditions and facilities, 
workload, and lack of recognition (Pearson & Seiler, 1983). 

A study by the Carnegie Foundation (cited in “Who Faculty,” 1985) 
indicated that 52.1% of the United States academics surveyed would serious- 
ly consider a reasonable offer for another academic appointment, and that 
47.0% would consider a reasonable offer in a nonacademic area. Nearly one 
fourth of those surveyed were reported to be considering entering another 
type of work. The reasons given were related to job dissatisfaction and 
salaries indicating a “deeply troubled profession” (Jacobson, 1985a). Jacob- 
son (1985b) stated that 45% of the academics in the Carnegie study would 
not recommend the professoriate to young people wishing to start a career. 
This was related to uneasiness of faculty members with regard to a perceived 
decline in academic standards and the erosion of job security, limited 
prospects for advancement, and personal stresses related to work. 

Barrett et al. (1983) have attributed dissatisfaction to the perceived dis- 
crepancy between an ideal set of circumstances and the realities of academic 
life. They cited the importance of teaching as an ideal from the academics’ 
points of view, and the lack of institutional importance placed on teaching as 
a reality, causing low morale. 

Much of the literature considers single-factor effects on job satisfaction. 
Hill (1986-1987) investigated Herzberg’s (1968) two-factor approach in rela- 
tion to academics’ satisfaction and dissatisfaction. Herzberg’s theory distin- 
guished between job content factors (intrinsic) that contribute to 
satisfaction, and job context factors (extrinsic) that contribute to dissatisfac- 
tion. The research supporting Herzberg’s approach has been mainly of the 
type that employs subjects’ descriptions of satisfying and dissatisfying work. 
As Gibson and Teasley (1973) point out, the validity of the theory has been 
questioned because support for it may have been contingent on the method- 
ology used. The simplicity of the model has been a source of criticism as well. 
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Hill (1986-1987) attempted to counter the criticism through use of a 
Likert-type questionnaire with faculty members from institutions of higher 
education. Factor analysis of the data produced six dimensions. Factor cor- 
relations indicated that there were two groups of three that were moderately 
related (.41 to .61) while the means of the two groups were significantly 
different (»<.01). Hill described the two groups as intrinsic and extrinsic, 
relating them to Herzberg’s theory. Within the groups the dimensions found 
were (a) intrinsic—teaching, recognition-support, and convenience; and (b) 
extrinsic—administrative, economic, and collegial. He recognized the limita- 
tions of this sample, but maintained that, to the extent that Pennsylvania 
academics were representative, the two-factor theory can be applied to in- 
stitutions of higher education. Further investigation of Hill’s contention 
seems necessary. 

As well, recent articles in the CAUT Bulletin expressing concerns of 
Canadian academics with regard to gender equity (Casey, 1986; Schrank, 
1989; Warren, 1986); security and tenure (Walker, 1983); need for career 
alternative programs (Renner, 1989); and impact of underfunding (Bal- 
zarini, 1989; Pfaffenberger, 1989) which are related to the satisfaction and 
dissatisfaction dimensions proposed by Hill (1986/1987) seem to indicate 
that research on Canadian academics’ career satisfaction is both warranted 
and timely. 


Purposes of the Study 
The purposes of this study were: 


1. To describe the types of activities engaged in by Canadian academics in 
relation to their preferences and institutional priorities. 

2. To compare level of general satisfaction of Canadian academics as 
measured by possibility of career change with choice of career and 
career progress. 

3. To determine positive and negative aspects of working in universities as 
stated by Canadian academics. 

4. To determine to what extent gender and rank relate to career satisfac- 
tion and progress, and to types of activities in which Canadian 
academics engage. 


Method 

A questionnaire was designed to provide information about the following: (a) 
the type of activities in which Canadian academics engage in relation to their 
preferences and institutional priorities; (b) the level of general satisfaction 
with career choice and progress; (c) the perceived positive and negative 
aspects of working in universities; and (d) the extent to which gender, age, 
rank, and income relate to career satisfaction and to types of work activities. 
The questionnaire was prepared in French and English forms. 

Astratified random sample of 997 Canadian academics was selected from 
the 1987 university calendars, representing the provinces by proportion to 
the overall number of academics employed (Statistics Canada, 1985). Each 
subject received a questionnaire, a cover letter asking for participation, and 
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a return envelope. Academics working in Francophone universities received 
a French translation of the questionnaire and the letter. 

A total of 447 questionnaires were returned. The responses were coded 
and cross tabulated. Percentages and chi squares were calculated and Yates 
corrections were applied where necessary using SPSS-x. 


Results and Discussion 

Questionnaires were returned by 447 academics, of whom 18.6% were 
women and 80.1% were men which compares favorably with the percentages 
(83% males and 17% females) given by the Association of Universities and 
Colleges of Canada (1984). The median age for the total group was 46 years. 
Most were married (75.8%) with three or fewer children living at home. 
Doctoral degrees had been earned by 76.1% of the group. Ninety-seven 
percent of the group were employed as full-time faculty members. There 
were 20.8% French questionnaires returned which is close to the 19.8% 
included in the initial sample. The ranks of associate professor (39.0%) and 
full professor (39.5%) were the most commonly held with 21.4% in the lower 
ranks. This was consistent with the Association of Universities and Colleges 
(1984) citation of 34% full professors, 36% associate professors, 20% assis- 
tant professors, and 10% others. The median number of years of university 
teaching was 16 with a median of 13.9 years in their present positions. 
Tenure had been granted to 81.1% with 50.4% being tenured for more than 
10 years. The median salary was $48,526. 

Care must be taken when comparing the sample of this study with that of 
the Carnegie study (Jacobson, 1985a, 1985b; “Who Faculty,” 1985) on 
United States college and university faculties. While the age range and the 
percentage of married faculty members appear to be similar, differences in 
other aspects of the sample are apparent. The United States sample had 
fewer in the upper ranks (58.6%), tenured (69.5%), and holding doctorates 
(57.5%). There were more females (27.0%) than in this study (18.6%). The 
median salaries for United States academics (US$28,000-$30,999) were 
lower but cannot be used in comparisons because of the 1985 date of the 
Carnegie study. 

When asked to identify the single activity that took the greatest amount 
of time in their present positions, 51.8% named teaching, with research at 
25.7%. Teaching (43.5%) and research (49.8%) were most frequently chosen 
as the most important activities from a personal standpoint while 83.3% 
named research and/or other professional production as the most important 
source of evidence for decisions about tenure in their faculties. 

It is interesting to note that percentages of time spent and personal 
importance given to an activity do not match the institutionally important 
activity percentages. In fact, 46% of the sample spent the greatest amount of 
time on activities other than those deemed most important personally. This 
may be a source of dissatisfaction as cited by Powell, Barrett, and Shanker 
(1983). If a person is assigned a workload involving a high percentage of time 
teaching, little time for research can be taken. Hill (1984) would anticipate 
that inconsistencies in expectations related to promotion, tenure, and 
general policy (external administrative dimension) would be a source of 


328 


Career Progress and Satisfaction 


dissatisfaction. However, some reduction in the stress caused by this might 
be expected in the current sample due to the high percentage of already 
tenured academics in the upper ranks. 

Several questions dealt with working closely with colleagues and stu- 
dents. Many (70.9%) had had a colleague who had acted as a mentor to them 
when they began their careers. As well, 76.6% stated that they act as mentors 
to their younger colleagues “sometimes” or “often.” Collaboration with col- 
leagues on professional production was quite frequent with only 15.8% stat- 
ing that they never collaborate. Collaboration with students on professional 
production was quite common also (62.4% in the “sometimes” and “often” 
categories). Instruction with colleagues had been experienced by 63.7% of the 
group. Generally, evidence of close working relationships with colleagues and 
students was found. 

When asked about time spent on administrative committees as compared 
with others with similar positions, 45.6% stated that they were above the 
average. When graduate committees were considered, 27.9% stated that they 
felt they spent more than the average amount of time. Many wrote com- 
ments indicating that there were no graduate students in their departments. 
Obviously a question on graduate/undergraduate offerings would have given 
a clearer indication of the working conditions represented here. 

When salary was rated in comparison to others in similar positions, 19.9% 
indicated that they believed their compensation to be below the average. 
Above average salaries were reported by 16.5%. 

The general level of satisfaction with career chosen was rated on a five- 
point scale. Those rating their feelings about choice of career as “satisfied” or 
“very satisfied” were 80.7% of the respondents, while 14.9% were “dissatis- 
fied” or “very dissatisfied.” When asked about career progress, 63.3% were 
satisfied or very satisfied while 13.9% were dissatisfied or very dissatisfied. 
Over 46% of the respondents reported that they had worked in more than one 
institution; of those, 53.8% had moved for greater professional opportunities. 
The percentage of respondents who would consider a career change at this 
point was 34.7%. 

The Carnegie data (cited in “Who Faculty,” 1985) indicated that 20.1% of 
their sample often wished they had entered another profession. Forty-seven 
percent stated they would consider a reasonable offer in a nonacademic 
position, while 52.1% stated that they would consider another academic 
position. While the data from this study do not indicate as high a percentage 
of dissatisfaction or consideration of career change as did the United States 
study, the percentages found are sufficiently high to cause concern, especially 
when consideration is given to the high proportion of faculty members who 
by their ranks appear to have been successful in university faculties. 

Faculty members were asked to state three major positive attributes of 
their work situation and three distressing aspects. The answers were 
categorized and tallied. Those listed below appeared on 12% or more of the 
questionnaires. The five most common positive attributes listed were (a) 
freedom, independence, or autonomy; (b) working with students; (c) collegial 
support and interaction; (d) research opportunities; and (e) stimulation and 
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personal growth. The six most common negative aspects were (a) ad- 
ministrative problems and lack of support; (b) lack of grants and funds; (c) 
workload; (d) hours and time pressures; (e) bureaucratic details; and (f) 
committee assignments. The positives and negatives are listed in order of 
highest to lowest occurrence with bureaucratic details and committee assign- 
ments named equally often in the negative category. 

When compared with the two-factor model of Hill (1986-1987), the col- 
legial support and interaction dimension is more of a positive aspect of the 
respondents’ work life than would have been predicted. The other dimen- 
sions mentioned by the respondents seem to fit Hill’s prediction that faculty 
members experience satisfaction with intrinsic factors and dissatisfaction 
with extrinsic factors related to their work. 

Cross tabulations were used to compare the answers on the question- 
naires with language of instruction, gender, rank, consideration of career 
change, and time spent doing preferred activities. Chi squares were com- 
puted (see Table 1) as were percentages. Statistically significant results 
(p<.05) are discussed below. 

The responses of academics from Francophone and Anglophone univer- 
sities were compared. There were no significant differences in age, gender, 
rank, year at rank, or salaries. The Anglophone group reported having 
mentors, holding PhDs, serving on administrative committees, and rating 
salaries below average more frequently than did the Francophones. More 
Francophones than Anglophones were tenured. 

There were no differences between the groups in satisfaction with chosen 
career or progress. It is interesting to note than 37.3% of the Anglophone 
respondents would consider a career change at this time, while only 24.0% of 
the Francophones would do so. Considering the possible factors related to 
this difference, one might speculate that lack of tenure and below average 
salaries account for a portion of the finding. Perceptions of potential for 
mobility were not investigated, nor were the careers to which the respon- 
dents would consider moving. 

Content analyses of responses indicating positive and negative aspects of 
working in universities produced few differences between Francophones and 
Anglophones. The groups did not differ in the positive aspects that were 
listed. They were as reported for the total sample. Some differences in the 
negative aspects did occur. In the Francophone group, the negative aspects 
most frequently stated in order of frequency were (a) lack of collegial support 
and interaction, (b) administrative problems and lack of support, (c) 
workload, (d) lack of resources, (e) low salaries, and (f) lack of recognition for 
teaching. Low salary and lack of recognition for teaching occurred equally 
frequently. The Anglophones listed the following: (a) administrative 
problems and lack of support, (b) workload, (c) lack of grants and other 
research funds, (d) low salaries, and (e) number of committee assignments. 

As in much of the literature on gender of academics, a greater proportion 
of women than men had part-time appointments, lower ranks, less time since 
tenure, fewer years of employment, lower degrees, and lower salaries. They 
tended to rate their salaries as “below average” for others with similar 
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positions in their institutions as well. Discrimination was cited more fre- 
quently by women (30.4%) than by men (22.6%) with sexual discrimination 
ranking as the most frequent type (86% of the responses). 

Women’s comments about the positive aspects of their work fell into the 
same four most frequent categories as did men’s: (a) freedom, independence, 
autonomy; (b) working with students; (c) collegial support and interaction; 
and (d) stimulation and personal growth. The fifth category was research for 
men and flexible schedules for women. The four most frequent categories for 
negative aspects about work were alike for women and men. They were (a) 
administrative problems and lack of support, (b) lack of grants and funds, (c) 
workload, and (d) negative collegial attitudes. The fifth ranking negative 
aspect for men was salary while women ranked hours and time pressures, 
bureaucratic details, and large classes equally often and in fifth position. It is 
interesting to note that only one woman (1.2%) listed salary as a negative 
aspect of her work while 52 men (15.3%) did so. The median salary for men 
was $49,840 while for women it was $44,008. 

When cross tabulations were done by rank, it was not surprising to find 
that full professors were found to be most likely to have been tenured, 
employed, and in university teaching for the longest amount of time when 
compared to the other ranks. They also acted as mentors, collaborated with 
students, spent more time in research, and instructed with colleagues more 
frequently than did others. The upper ranks reported the highest salaries, 
with full professors rating their salaries as average or above when compared 
to others with similar positions in 91.5% of the cases. Associate and full 
professors were the most satisfied with their positions, and full professors the 
most satisfied with their progress. 

When cross tabulations and chi squares were run with consideration of a 
career change at this time, lecturers were the groups with the highest propor- 
tion of positive responses. Most (80.1%) of those responding positively were 
male. Perceived discrimination was cited by 32.5% of the group with 11.9% 
related to promotion and 14.56% related to more than one area. As might 
have been predicted, satisfaction with career choice and career progress were 
highly related to lack of consideration of career change. Supervision of 
graduate students, administrative committee participation, and collabora- 
tion with students on professional production were all associated with no 
consideration of change. 

When asked to state three positive aspects of their work, those who would 
consider a career change listed the following (in order by frequency): (a) 
freedom, independence, and autonomy; (b) flexible schedules and time to do 
research; (c) collegial interaction and support; (d) working with students; 
and (e) stimulation and personal growth. While the faculty members who 
would consider a change most frequently listed the same positive aspects of 
their work as did those who would not consider a change, the order was 
slightly different. Those who would not consider a change when compared to 
the total did not differ in order by frequency of the aspects listed. 

The negative aspects most frequently listed by those who would consider 
a career change were (a) lack of funds for programs and research, (b) 
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administrative problems and lack of support, (c) poor collegial interaction 
and attitude, (d) heavy workload, (e) serving on committees, (f) bureaucratic 
details, and (g) salaries. Here again, the order by frequency is different from 
the sample as a whole and from the group which would not consider a change. 
Also, included with frequencies of 12% or more of those responding are 
collegial interaction and attitude and salaries which were not listed with such 
frequency in the total group. Those who would consider a change did not list 
hours and time pressures as frequently as did the group as a whole. The 
group which would not consider a change matched the total group in fre- 
quently listed negative aspects. Here as in the data cited above, Hill’s (1986- 
1987) interpretation of Herzberg’s approach was confirmed except for the 
inclusion of collegial interaction as positive, and consequently a contributor 
to satisfaction. 

The possible effect of academics spending most of their time on their 
preferred activities as compared to those spending time on their less 
preferred activities was investigated. Group I was comprised of those 
answers to questions about activities in which most time was spent matched 
their preferences (n=109). Group II’s time spent did not match with 
preferences (n=335). Cross tabulations and chi squares were run for the two 
groups. Group I was found to have fewer tenured, more in lower ranks, and 
longer times at rank than was Group II. Group I had been employed at the 
university level for a shorter time and made lower salaries than Group II. 
They were less likely to act as mentors or to instruct with colleagues than 
were those in Group IJ. There were no gender differences between the 
groups. It is interesting to note that the groups did not differ on career 
progress or general satisfaction, nor did they differ on consideration of career 
change which might have been expected from the statement of preference 
involved. 


Conclusions 

The Canadian academics responding to the questionnaire in this study ap- 
peared to be more satisfied with their positions and considered career change 
in proportionately fewer cases than was reported in the United States sample 
(“Who Faculty,” 1985). The differences may relate to a greater proportion of 
respondents in the higher ranks in the sample for this study as compared to 
that in the Carnegie study. A smaller percentage of respondents were women 
in this study than in the United States data. The pattern of responses 
reflected the literature on the status of women in academic positions in terms 
of lower ranks, lower pay, and more perceived discrimination. 

Consideration of career change was related to the stated positive and 
negative aspects of the respondents’ working situations. The data seem to 
support Hill’s (1986-1987) model, except in the collegial dimension of the 
extrinsic category. According to the two-factor theory, when faculty mem- 
bers experience dissatisfaction it is most likely to be concentrated in the 
extrinsic parts of their experience, while satisfaction is likely to be con- 
centrated in intrinsic areas. The area of collegial support, attitude, and 
interaction appears both as a strong positive and a strong negative in the 
current study. It is my contention that there is not a clear enough division 
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between the collegial dimension (extrinsic) and the recognition-support 
dimension (intrinsic) to categorize the statements of the subjects in a free 
response situation. This is consistent with Hill’s own description of excep- 
tions in his recognition-support dimension data. Further application and 
analysis of the model seems warranted. An interview format as a follow-up to 
questionnaire-generated data might prove quite useful in such a study. 
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